1.1.2
Click here for docs & updates
Created & maintained by Francois Bertrand
Graphic design by Jean-Francois Hains
DataFrame
NO COMPARISON TARGET
103
ROWS
0
DUPLICATES
78.0 kb
RAM
58
FEATURES
11
CATEGORICAL
45
NUMERICAL
2
TEXT
1
App ID
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
1
1%
-
-
app-20201104183249-0177
1
1%
-
-
app-20201104210518-0268
1
1%
-
-
app-20201104192439-0205
1
1%
-
-
app-20201104185756-0193
1
1%
-
-
app-20201104195352-0233
1
1%
-
-
app-20201104185956-0195
1
1%
-
-
app-20201104210007-0265
96
93%
-
-
(Other)
2
App Name
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
1
(1%)
3
query
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
1
1%
-
-
q83
1
1%
-
-
q47
1
1%
-
-
q95
1
1%
-
-
q27
1
1%
-
-
q34
1
1%
-
-
q62
1
1%
-
-
q1
96
93%
-
-
(Other)
4
AllSuccessful
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
2
(2%)
5
aLogSuccess
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
2
(2%)
6
AppSuccessful
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
2
(2%)
7
FailureCounts
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
3
(3%)
8
RunType
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
1
(1%)
9
aQueryTime
VALUES:
77
(75%)
MISSING:
26
(25%)
DISTINCT:
77
(75%)
MAX
1.0M
95%
0.3M
Q3
0.0M
AVG
0.1M
MEDIAN
0.0M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
1.0M
IQR
27,913
STD
130k
VAR
17.0B
KURT.
39.4
SKEW
5.74
SUM
4.9M
10
elapsedTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
1.1M
95%
0.3M
Q3
0.1M
AVG
0.1M
MEDIAN
0.1M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
1.0M
IQR
42,544
STD
121k
VAR
14.6B
KURT.
40.9
SKEW
5.63
SUM
9.6M
11
AppDurationSecs
VALUES:
101
(98%)
MISSING:
2
(2%)
DISTINCT:
101
(98%)
MAX
1,066
95%
308
Q3
95
AVG
99
MEDIAN
63
Q1
50
5%
37
MIN
36
RANGE
1,030
IQR
44.3
STD
122
VAR
14,961
KURT.
39.5
SKEW
5.52
SUM
9,995
12
stageDuration
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
1.5M
95%
0.5M
Q3
0.1M
AVG
0.1M
MEDIAN
0.1M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
1.5M
IQR
50,794
STD
229k
VAR
52.3B
KURT.
15.9
SKEW
3.67
SUM
13.6M
13
numStages
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
22
(21%)
MAX
63.0
95%
53.7
Q3
43.5
MEDIAN
40.0
AVG
39.9
Q1
36.0
5%
31.0
MIN
31.0
RANGE
32.0
IQR
7.50
STD
7.23
VAR
52.3
KURT.
0.837
SKEW
0.804
SUM
4,110
14
numTasks
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
73
(71%)
MAX
55,825
95%
21,696
Q3
15,877
AVG
15,364
MEDIAN
14,187
Q1
12,376
5%
12,063
MIN
12,063
RANGE
43,762
IQR
3,502
STD
5,120
VAR
26.2M
KURT.
38.4
SKEW
5.28
SUM
1.6M
15
executorRunTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
128.0M
95%
74.7M
Q3
23.7M
AVG
24.1M
MEDIAN
15.2M
Q1
11.7M
5%
8.9M
MIN
8.8M
RANGE
119.2M
IQR
12.0M
STD
23.4M
VAR
547.4T
KURT.
7.38
SKEW
2.65
SUM
2.5B
16
executorCpuTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
62.6M
95%
22.6M
Q3
3.0M
AVG
4.7M
MEDIAN
1.4M
Q1
0.6M
5%
0.5M
MIN
0.5M
RANGE
62.1M
IQR
2.4M
STD
9.5M
VAR
90.3T
KURT.
16.2
SKEW
3.74
SUM
488.0M
17
executorDeserializeTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
2.8M
95%
2.3M
Q3
2.2M
MEDIAN
2.0M
AVG
2.0M
Q1
1.7M
5%
1.7M
MIN
1.6M
RANGE
1.2M
IQR
428k
STD
225k
VAR
50.5B
KURT.
0.445
SKEW
0.310
SUM
204.2M
18
executorDeserializeCpuTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
873k
95%
554k
Q3
536k
MEDIAN
501k
AVG
488k
Q1
430k
5%
388k
MIN
374k
RANGE
499k
IQR
106k
STD
70,555
VAR
5.0B
KURT.
7.25
SKEW
1.12
SUM
50.3M
19
resultSerializationTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
96
(93%)
MAX
1,950
95%
1,231
Q3
892
AVG
789
MEDIAN
757
Q1
627
5%
414
MIN
282
RANGE
1,668
IQR
265
STD
266
VAR
70,921
KURT.
2.79
SKEW
1.07
SUM
81,307
20
jvmGCTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
22.1M
95%
5.1M
Q3
1.4M
AVG
1.7M
MEDIAN
0.9M
Q1
0.5M
5%
0.5M
MIN
0.5M
RANGE
21.6M
IQR
868k
STD
2.8M
VAR
8.0T
KURT.
32.9
SKEW
5.30
SUM
176.9M
21
shuffleFetchWaitTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
70
(68%)
MAX
6.2M
95%
0.9M
Q3
0.0M
AVG
0.2M
MEDIAN
0.0M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
6.2M
IQR
45,137
STD
671k
VAR
450.1B
KURT.
64.9
SKEW
7.51
SUM
17.6M
22
shuffleWriteTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
1.6M
95%
1.1M
Q3
0.1M
AVG
0.1M
MEDIAN
0.0M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
1.6M
IQR
95,372
STD
323k
VAR
104.3B
KURT.
10.3
SKEW
3.32
SUM
14.4M
23
resultSize
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
54
(52%)
MAX
816.5M
95%
24.5M
Q3
8.3M
AVG
16.0M
MEDIAN
4.9M
Q1
4.6M
5%
4.6M
MIN
4.6M
RANGE
812.0M
IQR
3.7M
STD
79.9M
VAR
6385.4T
KURT.
102
SKEW
10.1
SUM
1.6B
24
diskBytesSpilled
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
4
(4%)
25
memoryBytesSpilled
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
4
(4%)
26
peakExecutionMemory
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
72
(70%)
MAX
981.4B
95%
634.1B
Q3
80.8B
AVG
108.4B
MEDIAN
43.6B
Q1
0.1B
5%
0.0B
MIN
0.0B
RANGE
981.4B
IQR
80.7B
STD
194.8B
VAR
37930818826.1T
KURT.
7.57
SKEW
2.80
SUM
11.2T
27
recordsRead
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
74
(72%)
MAX
129.6B
95%
17.0B
Q3
2.3B
AVG
4.7B
MEDIAN
0.5B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
129.6B
IQR
2.3B
STD
15.5B
VAR
238735239.9T
KURT.
45.5
SKEW
6.32
SUM
480.5B
28
bytesRead
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
77
(75%)
MAX
340.1B
95%
118.8B
Q3
19.0B
AVG
25.8B
MEDIAN
4.3B
Q1
0.1B
5%
0.0B
MIN
0.0B
RANGE
340.1B
IQR
18.9B
STD
56.3B
VAR
3168824639.9T
KURT.
15.2
SKEW
3.64
SUM
2.7T
29
recordsWritten
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
21
(20%)
MAX
272k
95%
8k
Q3
0k
AVG
4k
MEDIAN
0k
Q1
0k
5%
0k
MIN
0k
RANGE
272k
IQR
100
STD
27,963
VAR
781.9M
KURT.
84.9
SKEW
8.94
SUM
438k
30
bytesWritten
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
77
(75%)
MAX
5.3M
95%
0.6M
Q3
0.0M
AVG
0.2M
MEDIAN
0.0M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
5.3M
IQR
5,202
STD
759k
VAR
576.1B
KURT.
32.0
SKEW
5.61
SUM
17.3M
31
shuffleRecordsRead
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
78
(76%)
MAX
39.1B
95%
8.4B
Q3
0.5B
AVG
1.6B
MEDIAN
0.0B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
39.1B
IQR
519.0M
STD
4.9B
VAR
23796331.2T
KURT.
36.6
SKEW
5.51
SUM
160.0B
32
shuffleTotalBlocksFetched
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
77
(75%)
MAX
2.6M
95%
1.2M
Q3
0.3M
AVG
0.3M
MEDIAN
0.1M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
2.6M
IQR
283k
STD
462k
VAR
213.1B
KURT.
8.49
SKEW
2.76
SUM
27.9M
33
shuffleLocalBlocksFetched
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
78
(76%)
MAX
20,118
95%
9,611
Q3
2,073
AVG
2,173
MEDIAN
836
Q1
14
5%
0
MIN
0
RANGE
20,118
IQR
2,059
STD
3,635
VAR
13.2M
KURT.
8.30
SKEW
2.73
SUM
224k
34
shuffleRemoteBlocksFetched
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
2.5M
95%
1.2M
Q3
0.3M
AVG
0.3M
MEDIAN
0.1M
Q1
0.0M
5%
0.0M
MIN
0.0M
RANGE
2.5M
IQR
281k
STD
458k
VAR
209.7B
KURT.
8.49
SKEW
2.76
SUM
27.6M
35
shuffleTotalBytesRead
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
482.2B
95%
175.7B
Q3
6.9B
AVG
29.9B
MEDIAN
0.5B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
482.2B
IQR
6.9B
STD
92.1B
VAR
8480805814.3T
KURT.
16.4
SKEW
4.05
SUM
3.1T
36
shuffleLocalBytesRead
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
3.8B
95%
1.4B
Q3
0.1B
AVG
0.2B
MEDIAN
0.0B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
3.8B
IQR
80.4M
STD
722.1M
VAR
521360.5T
KURT.
16.3
SKEW
4.03
SUM
24.9B
37
shuffleRemoteBytesRead
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
478.4B
95%
174.3B
Q3
6.6B
AVG
29.6B
MEDIAN
0.5B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
478.4B
IQR
6.6B
STD
91.4B
VAR
8348496080.5T
KURT.
16.4
SKEW
4.05
SUM
3.1T
38
shuffleRemoteBytesReadToDisk
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
1
(1%)
39
shuffleBytesWritten
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
318.5B
95%
174.5B
Q3
8.7B
AVG
22.2B
MEDIAN
0.5B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
318.5B
IQR
8.7B
STD
58.7B
VAR
3448357959.8T
KURT.
11.7
SKEW
3.43
SUM
2.3T
40
shuffleRecordsWritten
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
78
(76%)
MAX
14.3B
95%
8.8B
Q3
0.5B
AVG
1.2B
MEDIAN
0.0B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
14.3B
IQR
527.7M
STD
2.7B
VAR
7502109.9T
KURT.
7.25
SKEW
2.73
SUM
125.9B
41
submissionTime
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
103
(100%)
MAX
1604524.9M
95%
1604524.3M
Q3
1604522.4M
AVG
1604519.5M
MEDIAN
1604519.3M
Q1
1604517.2M
5%
1604515.2M
MIN
1604514.7M
RANGE
10.2M
IQR
5.2M
STD
3.0M
VAR
8.8T
KURT.
-1.10
SKEW
0.159
SUM
165.3T
42
cpuTimeRatio
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
66
(64%)
MAX
0.636
95%
0.410
Q3
0.122
AVG
0.131
MEDIAN
0.091
Q1
0.062
5%
0.049
MIN
0.027
RANGE
0.609
IQR
0.060
STD
0.117
VAR
0.014
KURT.
5.14
SKEW
2.32
SUM
13.5
43
ioTimeRatio
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
66
(64%)
MAX
0.973
95%
0.951
Q3
0.938
MEDIAN
0.909
AVG
0.869
Q1
0.877
5%
0.590
MIN
0.364
RANGE
0.609
IQR
0.060
STD
0.117
VAR
0.014
KURT.
5.14
SKEW
-2.32
SUM
89.5
44
jvmGCRatio
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
57
(55%)
MAX
0.184
95%
0.127
Q3
0.070
AVG
0.065
MEDIAN
0.058
Q1
0.049
5%
0.033
MIN
0.013
RANGE
0.171
IQR
0.021
STD
0.030
VAR
8.77e-4
KURT.
4.93
SKEW
1.92
SUM
6.67
45
shuffleFetchWaitTimeRatio
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
19
(18%)
MAX
0.048
95%
0.018
Q3
0.002
AVG
0.003
MEDIAN
0.000
Q1
0.000
5%
0.000
MIN
0.000
RANGE
0.048
IQR
0.002
STD
0.009
VAR
7.79e-5
KURT.
13.4
SKEW
3.57
SUM
0.358
46
totalBytesRead
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
590.3B
95%
336.5B
Q3
29.9B
AVG
55.7B
MEDIAN
8.1B
Q1
0.1B
5%
0.0B
MIN
0.0B
RANGE
590.3B
IQR
29.8B
STD
123.1B
VAR
15148847535.5T
KURT.
9.68
SKEW
3.11
SUM
5.7T
47
totalBytesWritten
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
318.5B
95%
174.5B
Q3
8.7B
AVG
22.2B
MEDIAN
0.5B
Q1
0.0B
5%
0.0B
MIN
0.0B
RANGE
318.5B
IQR
8.7B
STD
58.7B
VAR
3448381899.7T
KURT.
11.7
SKEW
3.43
SUM
2.3T
48
diskBytesSpilledRatio
VALUES:
78
(76%)
MISSING:
25
(24%)
DISTINCT:
3
(3%)
49
memoryBytesSpilledRatio
VALUES:
78
(76%)
MISSING:
25
(24%)
DISTINCT:
4
(4%)
50
totalBytesIORatio
VALUES:
78
(76%)
MISSING:
25
(24%)
DISTINCT:
76
(74%)
MAX
395k
95%
5k
Q3
0k
AVG
11k
MEDIAN
0k
Q1
0k
5%
0k
MIN
0k
RANGE
395k
IQR
220
STD
57,003
VAR
3.2B
KURT.
35.7
SKEW
5.94
SUM
839k
51
shuffleBytesIORatio
VALUES:
78
(76%)
MISSING:
25
(24%)
DISTINCT:
30
(29%)
MAX
6.04
95%
1.96
Q3
1.02
AVG
1.15
MEDIAN
1.00
Q1
1.00
5%
1.00
MIN
0.07
RANGE
5.97
IQR
0.020
STD
0.661
VAR
0.437
KURT.
39.6
SKEW
5.56
SUM
89.8
52
elapsedTime_minutes
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
98
(95%)
MAX
17.7
95%
4.9
Q3
1.5
AVG
1.6
MEDIAN
1.0
Q1
0.8
5%
0.5
MIN
0.5
RANGE
17.2
IQR
0.710
STD
2.01
VAR
4.06
KURT.
40.9
SKEW
5.63
SUM
160
53
peakExecutionMemory_GB
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
72
(70%)
MAX
914
95%
591
Q3
75
AVG
101
MEDIAN
41
Q1
0
5%
0
MIN
0
RANGE
914
IQR
75.1
STD
181
VAR
32,900
KURT.
7.57
SKEW
2.80
SUM
10,395
54
totalBytesRead_GB
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
79
(77%)
MAX
550
95%
313
Q3
28
AVG
52
MEDIAN
8
Q1
0
5%
0
MIN
0
RANGE
550
IQR
27.7
STD
115
VAR
13,140
KURT.
9.68
SKEW
3.11
SUM
5,343
55
totalBytesWritten_GB
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
72
(70%)
MAX
297
95%
163
Q3
8
AVG
21
MEDIAN
0
Q1
0
5%
0
MIN
0
RANGE
297
IQR
8.07
STD
54.7
VAR
2,991
KURT.
11.7
SKEW
3.43
SUM
2,134
56
shuffleTotalBytesRead_GB
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
72
(70%)
MAX
449
95%
164
Q3
6
AVG
28
MEDIAN
0
Q1
0
5%
0
MIN
0
RANGE
449
IQR
6.39
STD
85.8
VAR
7,356
KURT.
16.4
SKEW
4.05
SUM
2,865
57
shuffleBytesWritten_GB
VALUES:
103
(100%)
MISSING:
---
DISTINCT:
72
(70%)
MAX
297
95%
163
Q3
8
AVG
21
MEDIAN
0
Q1
0
5%
0
MIN
0
RANGE
297
IQR
8.07
STD
54.7
VAR
2,991
KURT.
11.7
SKEW
3.43
SUM
2,134
58
sparkSetupTime
VALUES:
101
(98%)
MISSING:
2
(2%)
DISTINCT:
100
(97%)
MAX
18.7
95%
7.3
Q3
5.5
AVG
5.2
MEDIAN
4.7
Q1
4.4
5%
4.2
MIN
3.7
RANGE
15.0
IQR
1.08
STD
1.77
VAR
3.12
KURT.
34.6
SKEW
5.02
SUM
526
Associations
[Only including dataset "DataFrame"] Squares are categorical associations (uncertainty coefficient & correlation ratio) from 0 to 1. The uncertainty coefficient is assymmetrical, (approximating how much the elements on the left PROVIDE INFORMATION on elements in the row). Circles are numerical correlations (Pearson's) from -1 to 1. The trivial diagonal is intentionally left blank for clarity.
Associations
[Only including dataset "None"] Squares are categorical associations (uncertainty coefficient & correlation ratio) from 0 to 1. The uncertainty coefficient is assymmetrical, (approximating how much the elements on the left PROVIDE INFORMATION on elements in the row). Circles are numerical correlations (Pearson's) from -1 to 1. The trivial diagonal is intentionally left blank for clarity.
App ID
MISSING:
---
1
1%
-
-
app-20201104183249-0177
1
1%
-
-
app-20201104210518-0268
1
1%
-
-
app-20201104192439-0205
1
1%
-
-
app-20201104185756-0193
1
1%
-
-
app-20201104195352-0233
1
1%
-
-
app-20201104185956-0195
1
1%
-
-
app-20201104210007-0265
1
1%
-
-
app-20201104193559-0216
1
1%
-
-
app-20201104210734-0270
1
1%
-
-
app-20201104200039-0240
1
1%
-
-
app-20201104183351-0178
1
1%
-
-
app-20201104192346-0204
1
1%
-
-
app-20201104184200-0183
1
1%
-
-
app-20201104204512-0254
1
1%
-
-
app-20201104194231-0221
1
1%
-
-
app-20201104194323-0222
1
1%
-
-
app-20201104205059-0257
1
1%
-
-
app-20201104210121-0266
1
1%
-
-
app-20201104190534-0199
1
1%
-
-
app-20201104205905-0264
1
1%
-
-
app-20201104204654-0255
1
1%
-
-
app-20201104193701-0217
1
1%
-
-
app-20201104194630-0225
1
1%
-
-
app-20201104185617-0191
1
1%
-
-
app-20201104212128-0277
1
1%
-
-
app-20201104192254-0203
1
1%
-
-
app-20201104195113-0230
1
1%
-
-
app-20201104210628-0269
1
1%
-
-
app-20201104200155-0241
1
1%
-
-
app-20201104185901-0194
1
1%
-
-
app-20201104183926-0180
1
1%
-
-
app-20201104194816-0227
1
1%
-
-
app-20201104190454-0198
1
1%
-
-
app-20201104193149-0211
1
1%
-
-
app-20201104203119-0249
1
1%
-
-
app-20201104193246-0212
1
1%
-
-
app-20201104193521-0215
1
1%
-
-
app-20201104204056-0252
1
1%
-
-
app-20201104194153-0220
1
1%
-
-
app-20201104184806-0187
1
1%
-
-
app-20201104211323-0274
1
1%
-
-
app-20201104184926-0189
1
1%
-
-
app-20201104192155-0202
1
1%
-
-
app-20201104184844-0188
1
1%
-
-
app-20201104193010-0209
1
1%
-
-
app-20201104195537-0235
1
1%
-
-
app-20201104202759-0247
1
1%
-
-
app-20201104192900-0208
1
1%
-
-
app-20201104202657-0246
1
1%
-
-
app-20201104202927-0248
1
1%
-
-
app-20201104194927-0228
1
1%
-
-
app-20201104211151-0272
1
1%
-
-
app-20201104200647-0243
1
1%
-
-
app-20201104195440-0234
1
1%
-
-
app-20201104204803-0256
1
1%
-
-
app-20201104210418-0267
1
1%
-
-
app-20201104205720-0262
1
1%
-
-
app-20201104184122-0182
1
1%
-
-
app-20201104205607-0261
1
1%
-
-
app-20201104205442-0260
43
42%
-
-
(Other)
App Name
MISSING:
---
TOP CATEGORIES
TPC-DS Like Bench
103
100%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
App Name
PROVIDES INFORMATION ON...
RunType
1.00
shuffleRemoteBytesReadToDisk
1.00
aLogSuccess
0.00
AllSuccessful
0.00
diskBytesSpilled
-0.00
memoryBytesSpilled
-0.00
memoryBytesSpilledRatio
-0.00
diskBytesSpilledRatio
-0.00
AppSuccessful
-0.00
FailureCounts
-0.00

THESE FEATURES
GIVE INFORMATION
ON App Name:
AllSuccessful
1.00
aLogSuccess
1.00
AppSuccessful
1.00
FailureCounts
1.00
RunType
1.00
diskBytesSpilled
1.00
memoryBytesSpilled
1.00
shuffleRemoteBytesReadToDisk
1.00
diskBytesSpilledRatio
1.00
memoryBytesSpilledRatio
1.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
App Name
CORRELATION RATIO WITH...
aQueryTime
0.00
elapsedTime
0.00
AppDurationSecs
0.00
stageDuration
0.00
numStages
0.00
numTasks
0.00
executorRunTime
0.00
executorCpuTime
0.00
executorDeserializeTime
0.00
executorDeserializeCpuTime
0.00
resultSerializationTime
0.00
jvmGCTime
0.00
shuffleFetchWaitTime
0.00
shuffleWriteTime
0.00
query
MISSING:
---
1
1%
-
-
q83
1
1%
-
-
q47
1
1%
-
-
q95
1
1%
-
-
q27
1
1%
-
-
q34
1
1%
-
-
q62
1
1%
-
-
q1
1
1%
-
-
q38
1
1%
-
-
q24a
1
1%
-
-
q57
1
1%
-
-
q24b
1
1%
-
-
q92
1
1%
-
-
q19
1
1%
-
-
q40
1
1%
-
-
q76
1
1%
-
-
q81
1
1%
-
-
q20
1
1%
-
-
q14b
1
1%
-
-
q61
1
1%
-
-
q51
1
1%
-
-
q89
1
1%
-
-
q14a
1
1%
-
-
q55
1
1%
-
-
q28
1
1%
-
-
q97
1
1%
-
-
q82
1
1%
-
-
q91
1
1%
-
-
q65
1
1%
-
-
q63
1
1%
-
-
q39b
1
1%
-
-
q29
1
1%
-
-
q31
1
1%
-
-
q68
1
1%
-
-
q16
1
1%
-
-
q33
1
1%
-
-
q45
1
1%
-
-
q3
1
1%
-
-
q53
1
1%
-
-
q43
1
1%
-
-
q5
1
1%
-
-
q56
1
1%
-
-
q86
1
1%
-
-
q96
1
1%
-
-
q67
1
1%
-
-
q9
1
1%
-
-
q93
1
1%
-
-
q87
1
1%
-
-
q60
1
1%
-
-
q23b
1
1%
-
-
q26
1
1%
-
-
q21
1
1%
-
-
q7
1
1%
-
-
q10
1
1%
-
-
q85
1
1%
-
-
q98
1
1%
-
-
q8
1
1%
-
-
q42
1
1%
-
-
q90
1
1%
-
-
q84
1
1%
-
-
q50
43
42%
-
-
(Other)
AllSuccessful
MISSING:
---
TOP CATEGORIES
True
75
73%
False
28
27%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
AllSuccessful
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
shuffleRemoteBytesReadToDisk
1.00
aLogSuccess
0.88
FailureCounts
0.28
AppSuccessful
0.27
diskBytesSpilled
0.11
memoryBytesSpilled
0.11
memoryBytesSpilledRatio
0.11
diskBytesSpilledRatio
0.06

THESE FEATURES
GIVE INFORMATION
ON AllSuccessful:
aLogSuccess
0.85
FailureCounts
0.09
AppSuccessful
0.04
diskBytesSpilled
0.03
memoryBytesSpilled
0.03
memoryBytesSpilledRatio
0.03
diskBytesSpilledRatio
0.01
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AllSuccessful
CORRELATION RATIO WITH...
executorDeserializeCpuTime
0.74
executorDeserializeTime
0.74
numStages
0.64
shuffleBytesIORatio
0.59
numTasks
0.35
executorRunTime
0.33
shuffleLocalBlocksFetched
0.32
ioTimeRatio
0.31
cpuTimeRatio
0.31
shuffleTotalBlocksFetched
0.31
shuffleRemoteBlocksFetched
0.31
peakExecutionMemory
0.26
peakExecutionMemory_GB
0.26
totalBytesRead_GB
0.26
aLogSuccess
MISSING:
---
TOP CATEGORIES
True
77
75%
False
26
25%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
aLogSuccess
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
shuffleRemoteBytesReadToDisk
1.00
AllSuccessful
0.85
AppSuccessful
0.29
diskBytesSpilled
0.12
memoryBytesSpilled
0.12
memoryBytesSpilledRatio
0.12
FailureCounts
0.08
diskBytesSpilledRatio
0.05

THESE FEATURES
GIVE INFORMATION
ON aLogSuccess:
AllSuccessful
0.88
AppSuccessful
0.05
diskBytesSpilled
0.03
memoryBytesSpilled
0.03
memoryBytesSpilledRatio
0.03
FailureCounts
0.02
diskBytesSpilledRatio
0.01
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
CORRELATION RATIO WITH...
executorDeserializeCpuTime
0.77
executorDeserializeTime
0.75
numStages
0.66
shuffleBytesIORatio
0.64
numTasks
0.34
executorRunTime
0.31
shuffleLocalBlocksFetched
0.31
cpuTimeRatio
0.30
ioTimeRatio
0.30
shuffleTotalBlocksFetched
0.30
shuffleRemoteBlocksFetched
0.30
sparkSetupTime
0.26
peakExecutionMemory
0.25
peakExecutionMemory_GB
0.25
AppSuccessful
MISSING:
---
TOP CATEGORIES
True
101
98%
False
2
2%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
AppSuccessful
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
shuffleRemoteBytesReadToDisk
1.00
FailureCounts
0.22
aLogSuccess
0.05
AllSuccessful
0.04
diskBytesSpilled
0.00
memoryBytesSpilled
0.00
memoryBytesSpilledRatio
0.00
diskBytesSpilledRatio
0.00

THESE FEATURES
GIVE INFORMATION
ON AppSuccessful:
FailureCounts
0.43
aLogSuccess
0.29
AllSuccessful
0.27
diskBytesSpilled
0.01
memoryBytesSpilled
0.01
memoryBytesSpilledRatio
0.01
diskBytesSpilledRatio
0.00
App Name
-0.00
RunType
-0.00
shuffleRemoteBytesReadToDisk
-0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AppSuccessful
CORRELATION RATIO WITH...
sparkSetupTime
0.38
executorDeserializeCpuTime
0.20
executorDeserializeTime
0.19
numStages
0.17
shuffleBytesIORatio
0.16
AppDurationSecs
0.11
submissionTime
0.11
cpuTimeRatio
0.10
ioTimeRatio
0.10
numTasks
0.09
resultSerializationTime
0.09
jvmGCRatio
0.08
shuffleLocalBlocksFetched
0.08
shuffleTotalBlocksFetched
0.08
FailureCounts
MISSING:
---
TOP CATEGORIES
0
99
96%
1
3
3%
4
1
1%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
FailureCounts
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
shuffleRemoteBytesReadToDisk
1.00
AppSuccessful
0.43
diskBytesSpilled
0.22
memoryBytesSpilled
0.22
memoryBytesSpilledRatio
0.22
AllSuccessful
0.09
aLogSuccess
0.02
diskBytesSpilledRatio
0.01

THESE FEATURES
GIVE INFORMATION
ON FailureCounts:
AllSuccessful
0.28
AppSuccessful
0.22
diskBytesSpilled
0.20
memoryBytesSpilled
0.20
memoryBytesSpilledRatio
0.20
aLogSuccess
0.08
diskBytesSpilledRatio
0.00
App Name
-0.00
RunType
-0.00
shuffleRemoteBytesReadToDisk
-0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
FailureCounts
CORRELATION RATIO WITH...
sparkSetupTime
0.27
resultSerializationTime
0.23
numStages
0.16
shuffleBytesIORatio
0.16
executorDeserializeCpuTime
0.15
executorDeserializeTime
0.14
cpuTimeRatio
0.12
ioTimeRatio
0.12
peakExecutionMemory_GB
0.12
peakExecutionMemory
0.12
AppDurationSecs
0.12
shuffleRecordsWritten
0.12
submissionTime
0.12
bytesWritten
0.10
RunType
MISSING:
---
TOP CATEGORIES
cpu
103
100%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
RunType
PROVIDES INFORMATION ON...
App Name
1.00
shuffleRemoteBytesReadToDisk
1.00
aLogSuccess
0.00
AllSuccessful
0.00
diskBytesSpilled
-0.00
memoryBytesSpilled
-0.00
memoryBytesSpilledRatio
-0.00
diskBytesSpilledRatio
-0.00
AppSuccessful
-0.00
FailureCounts
-0.00

THESE FEATURES
GIVE INFORMATION
ON RunType:
App Name
1.00
AllSuccessful
1.00
aLogSuccess
1.00
AppSuccessful
1.00
FailureCounts
1.00
diskBytesSpilled
1.00
memoryBytesSpilled
1.00
shuffleRemoteBytesReadToDisk
1.00
diskBytesSpilledRatio
1.00
memoryBytesSpilledRatio
1.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
RunType
CORRELATION RATIO WITH...
aQueryTime
0.00
elapsedTime
0.00
AppDurationSecs
0.00
stageDuration
0.00
numStages
0.00
numTasks
0.00
executorRunTime
0.00
executorCpuTime
0.00
executorDeserializeTime
0.00
executorDeserializeCpuTime
0.00
resultSerializationTime
0.00
jvmGCTime
0.00
shuffleFetchWaitTime
0.00
shuffleWriteTime
0.00
aQueryTime
MISSING:
26
(25%)
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
elapsedTime
0.99
elapsedTime_minutes
0.99
AppDurationSecs
0.98
ioTimeRatio
-0.70
cpuTimeRatio
0.70
stageDuration
0.65
executorCpuTime
0.60
peakExecutionMemory
0.50
peakExecutionMemory_GB
0.50
shuffleBytesWritten
0.47
shuffleBytesWritten_GB
0.47
totalBytesWritten
0.47
totalBytesWritten_GB
0.47
shuffleRemoteBytesRead
0.47

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.87
memoryBytesSpilled
0.87
memoryBytesSpilledRatio
0.87
diskBytesSpilledRatio
0.87
AllSuccessful
0.25
aLogSuccess
0.24
FailureCounts
0.07
AppSuccessful
0.06
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
24713.0
1
1.3%
360612.0
1
1.3%
21464.0
1
1.3%
22814.0
1
1.3%
22433.0
1
1.3%
82152.0
1
1.3%
45613.0
1
1.3%
26034.0
1
1.3%
34467.0
1
1.3%
12559.0
1
1.3%
16904.0
1
1.3%
27588.0
1
1.3%
150381.0
1
1.3%
19845.0
1
1.3%
57788.0
1
1.3%
SMALLEST VALUES
10254.0
1
1.3%
10466.0
1
1.3%
12559.0
1
1.3%
12869.0
1
1.3%
13498.0
1
1.3%
14161.0
1
1.3%
14739.0
1
1.3%
14756.0
1
1.3%
14808.0
1
1.3%
14964.0
1
1.3%
15705.0
1
1.3%
16771.0
1
1.3%
16806.0
1
1.3%
16904.0
1
1.3%
18615.0
1
1.3%
LARGEST VALUES
1024259.0
1
1.3%
360612.0
1
1.3%
307378.0
1
1.3%
283322.0
1
1.3%
271098.0
1
1.3%
166514.0
1
1.3%
150381.0
1
1.3%
149280.0
1
1.3%
130403.0
1
1.3%
112990.0
1
1.3%
102429.0
1
1.3%
82152.0
1
1.3%
78156.0
1
1.3%
77737.0
1
1.3%
57965.0
1
1.3%
elapsedTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
elapsedTime_minutes
1.00
AppDurationSecs
1.00
aQueryTime
0.99
cpuTimeRatio
0.71
ioTimeRatio
-0.71
stageDuration
0.68
executorCpuTime
0.63
peakExecutionMemory
0.56
peakExecutionMemory_GB
0.56
executorRunTime
0.51
shuffleRecordsWritten
0.50
shuffleBytesWritten
0.48
totalBytesWritten
0.48
shuffleBytesWritten_GB
0.48

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.86
memoryBytesSpilled
0.86
memoryBytesSpilledRatio
0.86
diskBytesSpilledRatio
0.82
AllSuccessful
0.21
aLogSuccess
0.20
FailureCounts
0.10
AppSuccessful
0.04
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
141820
1
1.0%
110172
1
1.0%
45892
1
1.0%
62790
1
1.0%
89310
1
1.0%
40779
1
1.0%
54350
1
1.0%
186192
1
1.0%
78418
1
1.0%
32204
1
1.0%
33624
1
1.0%
59994
1
1.0%
57950
1
1.0%
54845
1
1.0%
55143
1
1.0%
SMALLEST VALUES
32204
1
1.0%
32211
1
1.0%
32414
1
1.0%
32533
1
1.0%
32753
1
1.0%
32884
1
1.0%
32976
1
1.0%
33060
1
1.0%
33130
1
1.0%
33228
1
1.0%
33241
1
1.0%
33250
1
1.0%
33252
1
1.0%
33624
1
1.0%
34140
1
1.0%
LARGEST VALUES
1061285
1
1.0%
401051
1
1.0%
365847
1
1.0%
339480
1
1.0%
316551
1
1.0%
301950
1
1.0%
240372
1
1.0%
209672
1
1.0%
205111
1
1.0%
197414
1
1.0%
186192
1
1.0%
169614
1
1.0%
169399
1
1.0%
145453
1
1.0%
141820
1
1.0%
AppDurationSecs
MISSING:
2
(2%)
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
elapsedTime_minutes
1.00
elapsedTime
1.00
aQueryTime
0.98
cpuTimeRatio
0.71
ioTimeRatio
-0.71
stageDuration
0.68
executorCpuTime
0.63
peakExecutionMemory
0.56
peakExecutionMemory_GB
0.56
executorRunTime
0.51
shuffleRecordsWritten
0.50
shuffleBytesWritten
0.48
totalBytesWritten
0.48
totalBytesWritten_GB
0.48

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.85
memoryBytesSpilled
0.85
memoryBytesSpilledRatio
0.85
diskBytesSpilledRatio
0.82
AllSuccessful
0.23
aLogSuccess
0.22
FailureCounts
0.12
AppSuccessful
0.11
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
36.871
1
1.0%
58.98
1
1.0%
36.947
1
1.0%
65.304
1
1.0%
103.77
1
1.0%
37.382
1
1.0%
71.891
1
1.0%
60.291
1
1.0%
193.47
1
1.0%
53.45
1
1.0%
93.596
1
1.0%
49.532
1
1.0%
83.886
1
1.0%
44.738
1
1.0%
51.093
1
1.0%
SMALLEST VALUES
36.442
1
1.0%
36.644
1
1.0%
36.701
1
1.0%
36.871
1
1.0%
36.947
1
1.0%
37.156
1
1.0%
37.176
1
1.0%
37.382
1
1.0%
37.52
1
1.0%
37.609
1
1.0%
37.637
1
1.0%
37.827
1
1.0%
38.154
1
1.0%
38.455
1
1.0%
39.998
1
1.0%
LARGEST VALUES
1066.025
1
1.0%
405.947
1
1.0%
369.554
1
1.0%
344.549
1
1.0%
323.006
1
1.0%
307.974
1
1.0%
247.17
1
1.0%
228.417
1
1.0%
209.738
1
1.0%
207.242
1
1.0%
193.47
1
1.0%
175.069
1
1.0%
174.266
1
1.0%
149.968
1
1.0%
149.656
1
1.0%
stageDuration
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
executorRunTime
0.87
recordsRead
0.76
bytesRead
0.74
totalBytesRead_GB
0.69
totalBytesRead
0.69
elapsedTime
0.68
elapsedTime_minutes
0.68
AppDurationSecs
0.68
executorCpuTime
0.67
numStages
0.65
aQueryTime
0.65
jvmGCTime
0.56
totalBytesIORatio
0.56
cpuTimeRatio
0.54

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.53
memoryBytesSpilled
0.53
memoryBytesSpilledRatio
0.53
diskBytesSpilledRatio
0.50
AllSuccessful
0.24
aLogSuccess
0.23
FailureCounts
0.06
AppSuccessful
0.06
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
23551
1
1.0%
426845
1
1.0%
39495
1
1.0%
81736
1
1.0%
62283
1
1.0%
38988
1
1.0%
21582
1
1.0%
20815
1
1.0%
48209
1
1.0%
40274
1
1.0%
52307
1
1.0%
68213
1
1.0%
107108
1
1.0%
152388
1
1.0%
21860
1
1.0%
SMALLEST VALUES
20815
1
1.0%
20966
1
1.0%
20986
1
1.0%
21249
1
1.0%
21306
1
1.0%
21422
1
1.0%
21440
1
1.0%
21582
1
1.0%
21670
1
1.0%
21775
1
1.0%
21860
1
1.0%
21867
1
1.0%
21874
1
1.0%
22152
1
1.0%
22660
1
1.0%
LARGEST VALUES
1519628
1
1.0%
1049340
1
1.0%
830704
1
1.0%
745530
1
1.0%
739202
1
1.0%
471497
1
1.0%
453106
1
1.0%
443703
1
1.0%
435396
1
1.0%
426845
1
1.0%
426703
1
1.0%
413381
1
1.0%
325888
1
1.0%
237934
1
1.0%
208400
1
1.0%
numStages
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
executorRunTime
0.73
stageDuration
0.65
shuffleTotalBlocksFetched
0.65
shuffleRemoteBlocksFetched
0.65
shuffleLocalBlocksFetched
0.65
numTasks
0.59
totalBytesRead_GB
0.59
totalBytesRead
0.59
bytesRead
0.59
executorDeserializeCpuTime
0.55
executorCpuTime
0.53
recordsRead
0.51
executorDeserializeTime
0.51
shuffleFetchWaitTimeRatio
0.50

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
0.66
AllSuccessful
0.64
diskBytesSpilled
0.27
memoryBytesSpilled
0.27
memoryBytesSpilledRatio
0.27
diskBytesSpilledRatio
0.24
AppSuccessful
0.17
FailureCounts
0.16
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
31
25
24.3%
38
13
12.6%
42
9
8.7%
40
8
7.8%
36
7
6.8%
44
5
4.9%
41
5
4.9%
43
5
4.9%
48
5
4.9%
45
4
3.9%
37
3
2.9%
50
2
1.9%
39
2
1.9%
57
2
1.9%
46
1
1.0%
SMALLEST VALUES
31
25
24.3%
36
7
6.8%
37
3
2.9%
38
13
12.6%
39
2
1.9%
40
8
7.8%
41
5
4.9%
42
9
8.7%
43
5
4.9%
44
5
4.9%
45
4
3.9%
46
1
1.0%
47
1
1.0%
48
5
4.9%
49
1
1.0%
LARGEST VALUES
63
1
1.0%
62
1
1.0%
57
2
1.9%
56
1
1.0%
54
1
1.0%
51
1
1.0%
50
2
1.9%
49
1
1.0%
48
5
4.9%
47
1
1.0%
46
1
1.0%
45
4
3.9%
44
5
4.9%
43
5
4.9%
42
9
8.7%
numTasks
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
resultSize
0.80
shuffleBytesIORatio
0.76
executorDeserializeCpuTime
0.70
shuffleFetchWaitTimeRatio
0.63
numStages
0.59
peakExecutionMemory_GB
0.58
peakExecutionMemory
0.58
executorDeserializeTime
0.55
shuffleLocalBlocksFetched
0.50
executorRunTime
0.48
shuffleTotalBlocksFetched
0.47
shuffleRemoteBlocksFetched
0.47
stageDuration
0.44
bytesRead
0.40

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AllSuccessful
0.35
aLogSuccess
0.34
diskBytesSpilled
0.16
memoryBytesSpilled
0.16
memoryBytesSpilledRatio
0.16
diskBytesSpilledRatio
0.13
AppSuccessful
0.09
FailureCounts
0.07
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
12063
25
24.3%
13115
2
1.9%
18834
2
1.9%
13624
2
1.9%
13297
2
1.9%
16885
2
1.9%
15732
2
1.9%
15000
1
1.0%
14765
1
1.0%
15406
1
1.0%
15663
1
1.0%
21497
1
1.0%
21938
1
1.0%
15927
1
1.0%
29734
1
1.0%
SMALLEST VALUES
12063
25
24.3%
12320
1
1.0%
12431
1
1.0%
12735
1
1.0%
13033
1
1.0%
13115
2
1.9%
13297
2
1.9%
13306
1
1.0%
13402
1
1.0%
13415
1
1.0%
13425
1
1.0%
13426
1
1.0%
13437
1
1.0%
13503
1
1.0%
13603
1
1.0%
LARGEST VALUES
55825
1
1.0%
29734
1
1.0%
24939
1
1.0%
21938
1
1.0%
21935
1
1.0%
21718
1
1.0%
21497
1
1.0%
21189
1
1.0%
20349
1
1.0%
20332
1
1.0%
20317
1
1.0%
20112
1
1.0%
20089
1
1.0%
18834
2
1.9%
16885
2
1.9%
executorRunTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
stageDuration
0.87
totalBytesRead_GB
0.82
totalBytesRead
0.82
executorCpuTime
0.81
bytesRead
0.79
jvmGCTime
0.76
numStages
0.73
shuffleTotalBlocksFetched
0.70
shuffleRemoteBlocksFetched
0.70
shuffleLocalBlocksFetched
0.70
shuffleWriteTime
0.66
recordsRead
0.66
shuffleRecordsRead
0.64
shuffleRecordsWritten
0.63

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.46
memoryBytesSpilled
0.46
memoryBytesSpilledRatio
0.46
diskBytesSpilledRatio
0.45
AllSuccessful
0.33
aLogSuccess
0.31
AppSuccessful
0.08
FailureCounts
0.07
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
77443817
1
1.0%
18104918
1
1.0%
35226175
1
1.0%
12109831
1
1.0%
16862787
1
1.0%
8832581
1
1.0%
13670982
1
1.0%
8939336
1
1.0%
16168297
1
1.0%
10460130
1
1.0%
9958993
1
1.0%
94664152
1
1.0%
8891481
1
1.0%
26612597
1
1.0%
10734682
1
1.0%
SMALLEST VALUES
8804884
1
1.0%
8813799
1
1.0%
8832581
1
1.0%
8837134
1
1.0%
8849050
1
1.0%
8880760
1
1.0%
8886698
1
1.0%
8891481
1
1.0%
8939336
1
1.0%
9023955
1
1.0%
9053594
1
1.0%
9109554
1
1.0%
9127901
1
1.0%
9134766
1
1.0%
9143993
1
1.0%
LARGEST VALUES
128030905
1
1.0%
119974725
1
1.0%
107993833
1
1.0%
94664152
1
1.0%
77443817
1
1.0%
75368457
1
1.0%
68820285
1
1.0%
58073533
1
1.0%
56206374
1
1.0%
55491347
1
1.0%
54693913
1
1.0%
50974836
1
1.0%
45894371
1
1.0%
44491509
1
1.0%
43634293
1
1.0%
executorCpuTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
cpuTimeRatio
0.86
ioTimeRatio
-0.86
shuffleRecordsRead
0.85
shuffleRecordsWritten
0.84
shuffleWriteTime
0.83
jvmGCTime
0.82
executorRunTime
0.81
shuffleTotalBytesRead
0.80
shuffleTotalBytesRead_GB
0.80
shuffleRemoteBytesRead
0.80
shuffleLocalBytesRead
0.80
shuffleBytesWritten
0.79
shuffleBytesWritten_GB
0.79
totalBytesWritten
0.79

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.65
memoryBytesSpilled
0.65
memoryBytesSpilledRatio
0.65
diskBytesSpilledRatio
0.63
AllSuccessful
0.23
aLogSuccess
0.21
AppSuccessful
0.06
FailureCounts
0.06
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
1467135
1
1.0%
1291620
1
1.0%
554060
1
1.0%
544846
1
1.0%
1834033
1
1.0%
973649
1
1.0%
1474898
1
1.0%
1539902
1
1.0%
30612310
1
1.0%
528471
1
1.0%
1948761
1
1.0%
973408
1
1.0%
546917
1
1.0%
553033
1
1.0%
5265728
1
1.0%
SMALLEST VALUES
515047
1
1.0%
526739
1
1.0%
528471
1
1.0%
537419
1
1.0%
543671
1
1.0%
544242
1
1.0%
544846
1
1.0%
545732
1
1.0%
545777
1
1.0%
546917
1
1.0%
547602
1
1.0%
547804
1
1.0%
548874
1
1.0%
549386
1
1.0%
549413
1
1.0%
LARGEST VALUES
62594229
1
1.0%
43292881
1
1.0%
32436329
1
1.0%
30612310
1
1.0%
27770268
1
1.0%
22625907
1
1.0%
22403868
1
1.0%
22267662
1
1.0%
19651644
1
1.0%
18766449
1
1.0%
8278276
1
1.0%
7589427
1
1.0%
7569657
1
1.0%
7156846
1
1.0%
7003180
1
1.0%
executorDeserializeTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
executorDeserializeCpuTime
0.95
numTasks
0.55
shuffleBytesIORatio
0.51
numStages
0.51
resultSerializationTime
0.46
peakExecutionMemory_GB
0.40
peakExecutionMemory
0.40
resultSize
0.40
shuffleFetchWaitTimeRatio
0.37
shuffleLocalBlocksFetched
0.36
shuffleTotalBlocksFetched
0.35
shuffleRemoteBlocksFetched
0.35
jvmGCRatio
0.34
cpuTimeRatio
0.30

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
0.75
AllSuccessful
0.74
AppSuccessful
0.19
FailureCounts
0.14
diskBytesSpilled
0.08
memoryBytesSpilled
0.08
memoryBytesSpilledRatio
0.08
diskBytesSpilledRatio
0.08
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
1975551
1
1.0%
1693849
1
1.0%
1633368
1
1.0%
2242547
1
1.0%
1895774
1
1.0%
2089840
1
1.0%
1797730
1
1.0%
2169443
1
1.0%
1669990
1
1.0%
2091118
1
1.0%
2024304
1
1.0%
1963890
1
1.0%
1838453
1
1.0%
1924996
1
1.0%
1994359
1
1.0%
SMALLEST VALUES
1633368
1
1.0%
1651399
1
1.0%
1653127
1
1.0%
1661662
1
1.0%
1662140
1
1.0%
1669320
1
1.0%
1669990
1
1.0%
1674566
1
1.0%
1678664
1
1.0%
1679029
1
1.0%
1680151
1
1.0%
1683372
1
1.0%
1683842
1
1.0%
1687777
1
1.0%
1689906
1
1.0%
LARGEST VALUES
2835885
1
1.0%
2296069
1
1.0%
2290384
1
1.0%
2281284
1
1.0%
2278546
1
1.0%
2276860
1
1.0%
2272896
1
1.0%
2268625
1
1.0%
2263425
1
1.0%
2258495
1
1.0%
2251374
1
1.0%
2244367
1
1.0%
2242547
1
1.0%
2238116
1
1.0%
2237432
1
1.0%
executorDeserializeCpuTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
executorDeserializeTime
0.95
shuffleBytesIORatio
0.76
numTasks
0.70
resultSize
0.56
numStages
0.55
resultSerializationTime
0.51
peakExecutionMemory_GB
0.49
peakExecutionMemory
0.49
shuffleFetchWaitTimeRatio
0.46
shuffleLocalBlocksFetched
0.38
shuffleTotalBlocksFetched
0.36
shuffleRemoteBlocksFetched
0.36
cpuTimeRatio
0.29
ioTimeRatio
-0.29

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
0.77
AllSuccessful
0.74
AppSuccessful
0.20
FailureCounts
0.15
diskBytesSpilled
0.08
memoryBytesSpilled
0.08
memoryBytesSpilledRatio
0.08
diskBytesSpilledRatio
0.08
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
517630
1
1.0%
535605
1
1.0%
514747
1
1.0%
482124
1
1.0%
495694
1
1.0%
538703
1
1.0%
521556
1
1.0%
392496
1
1.0%
392282
1
1.0%
505691
1
1.0%
394076
1
1.0%
507229
1
1.0%
538216
1
1.0%
522312
1
1.0%
535306
1
1.0%
SMALLEST VALUES
373529
1
1.0%
373776
1
1.0%
377268
1
1.0%
384694
1
1.0%
385960
1
1.0%
387530
1
1.0%
389387
1
1.0%
389397
1
1.0%
390123
1
1.0%
390567
1
1.0%
392282
1
1.0%
392471
1
1.0%
392496
1
1.0%
392847
1
1.0%
392870
1
1.0%
LARGEST VALUES
872777
1
1.0%
572627
1
1.0%
564483
1
1.0%
562160
1
1.0%
556965
1
1.0%
553957
1
1.0%
553086
1
1.0%
550274
1
1.0%
549669
1
1.0%
548609
1
1.0%
547538
1
1.0%
547535
1
1.0%
544710
1
1.0%
542402
1
1.0%
542263
1
1.0%
resultSerializationTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleBytesIORatio
0.52
executorDeserializeCpuTime
0.51
executorDeserializeTime
0.46
resultSize
0.44
shuffleFetchWaitTimeRatio
0.42
numTasks
0.35
peakExecutionMemory_GB
0.29
peakExecutionMemory
0.29
totalBytesIORatio
-0.25
shuffleFetchWaitTime
0.20
submissionTime
-0.20
sparkSetupTime
-0.18
recordsWritten
0.17
numStages
0.16

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
0.23
FailureCounts
0.23
AllSuccessful
0.15
diskBytesSpilled
0.15
memoryBytesSpilled
0.15
memoryBytesSpilledRatio
0.15
diskBytesSpilledRatio
0.14
AppSuccessful
0.09
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
640
2
1.9%
816
2
1.9%
730
2
1.9%
618
2
1.9%
879
2
1.9%
679
2
1.9%
633
2
1.9%
693
1
1.0%
565
1
1.0%
694
1
1.0%
1183
1
1.0%
889
1
1.0%
929
1
1.0%
677
1
1.0%
1126
1
1.0%
SMALLEST VALUES
282
1
1.0%
328
1
1.0%
332
1
1.0%
352
1
1.0%
356
1
1.0%
413
1
1.0%
419
1
1.0%
444
1
1.0%
457
1
1.0%
485
1
1.0%
500
1
1.0%
524
1
1.0%
525
1
1.0%
539
1
1.0%
546
1
1.0%
LARGEST VALUES
1950
1
1.0%
1472
1
1.0%
1362
1
1.0%
1361
1
1.0%
1297
1
1.0%
1236
1
1.0%
1183
1
1.0%
1166
1
1.0%
1156
1
1.0%
1147
1
1.0%
1139
1
1.0%
1126
1
1.0%
1097
1
1.0%
1059
1
1.0%
1053
1
1.0%
jvmGCTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
executorCpuTime
0.82
executorRunTime
0.76
shuffleFetchWaitTime
0.76
shuffleWriteTime
0.68
jvmGCRatio
0.67
shuffleLocalBlocksFetched
0.64
shuffleTotalBlocksFetched
0.64
shuffleRemoteBlocksFetched
0.64
shuffleRecordsRead
0.64
shuffleRecordsWritten
0.62
totalBytesRead_GB
0.57
totalBytesRead
0.57
cpuTimeRatio
0.57
ioTimeRatio
-0.57

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.54
memoryBytesSpilled
0.54
memoryBytesSpilledRatio
0.54
diskBytesSpilledRatio
0.54
AllSuccessful
0.24
aLogSuccess
0.23
AppSuccessful
0.06
FailureCounts
0.04
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
668415
1
1.0%
902742
1
1.0%
853058
1
1.0%
5095491
1
1.0%
5957189
1
1.0%
2004039
1
1.0%
541768
1
1.0%
4323657
1
1.0%
529232
1
1.0%
703569
1
1.0%
509400
1
1.0%
2604884
1
1.0%
4051289
1
1.0%
659513
1
1.0%
1233673
1
1.0%
SMALLEST VALUES
487592
1
1.0%
488800
1
1.0%
490200
1
1.0%
490752
1
1.0%
496568
1
1.0%
498024
1
1.0%
503288
1
1.0%
503376
1
1.0%
504808
1
1.0%
506736
1
1.0%
507048
1
1.0%
507832
1
1.0%
508176
1
1.0%
509400
1
1.0%
512408
1
1.0%
LARGEST VALUES
22108736
1
1.0%
16422547
1
1.0%
6368251
1
1.0%
5957189
1
1.0%
5626482
1
1.0%
5095491
1
1.0%
4965384
1
1.0%
4604858
1
1.0%
4323657
1
1.0%
4051289
1
1.0%
3479031
1
1.0%
3180081
1
1.0%
3079438
1
1.0%
3007619
1
1.0%
2720387
1
1.0%
shuffleFetchWaitTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleRecordsRead
0.86
executorCpuTime
0.79
jvmGCTime
0.76
shuffleFetchWaitTimeRatio
0.76
shuffleRecordsWritten
0.66
shuffleLocalBytesRead
0.66
shuffleTotalBytesRead
0.66
shuffleTotalBytesRead_GB
0.66
shuffleRemoteBytesRead
0.66
executorRunTime
0.62
totalBytesRead
0.59
totalBytesRead_GB
0.59
shuffleLocalBlocksFetched
0.59
shuffleTotalBlocksFetched
0.58

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.89
memoryBytesSpilled
0.89
memoryBytesSpilledRatio
0.89
diskBytesSpilledRatio
0.89
AllSuccessful
0.14
aLogSuccess
0.14
AppSuccessful
0.04
FailureCounts
0.03
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
31
30.1%
5
2
1.9%
6
2
1.9%
11
2
1.9%
49585
1
1.0%
34
1
1.0%
36
1
1.0%
1214
1
1.0%
99732
1
1.0%
41
1
1.0%
689
1
1.0%
4527
1
1.0%
947
1
1.0%
3637
1
1.0%
566
1
1.0%
SMALLEST VALUES
0
31
30.1%
2
1
1.0%
5
2
1.9%
6
2
1.9%
9
1
1.0%
11
2
1.9%
15
1
1.0%
34
1
1.0%
36
1
1.0%
41
1
1.0%
69
1
1.0%
267
1
1.0%
287
1
1.0%
489
1
1.0%
566
1
1.0%
LARGEST VALUES
6187993
1
1.0%
2003701
1
1.0%
1134370
1
1.0%
1101835
1
1.0%
975459
1
1.0%
958464
1
1.0%
849867
1
1.0%
702329
1
1.0%
549870
1
1.0%
437759
1
1.0%
346813
1
1.0%
327525
1
1.0%
298849
1
1.0%
273375
1
1.0%
174170
1
1.0%
shuffleWriteTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleBytesWritten
0.97
shuffleBytesWritten_GB
0.97
totalBytesWritten
0.97
totalBytesWritten_GB
0.97
shuffleLocalBytesRead
0.91
shuffleTotalBytesRead
0.91
shuffleTotalBytesRead_GB
0.91
shuffleRemoteBytesRead
0.91
shuffleRecordsWritten
0.85
totalBytesRead_GB
0.84
totalBytesRead
0.84
executorCpuTime
0.83
shuffleRecordsRead
0.77
cpuTimeRatio
0.77

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.31
memoryBytesSpilled
0.31
memoryBytesSpilledRatio
0.31
diskBytesSpilledRatio
0.31
AllSuccessful
0.25
aLogSuccess
0.24
AppSuccessful
0.06
FailureCounts
0.06
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
12199
1
1.0%
13854
1
1.0%
78111
1
1.0%
2208
1
1.0%
211873
1
1.0%
36770
1
1.0%
3108
1
1.0%
1122213
1
1.0%
1367337
1
1.0%
23641
1
1.0%
70792
1
1.0%
564162
1
1.0%
4909
1
1.0%
27182
1
1.0%
SMALLEST VALUES
0
25
24.3%
912
1
1.0%
2208
1
1.0%
2452
1
1.0%
3108
1
1.0%
3219
1
1.0%
4909
1
1.0%
5053
1
1.0%
6140
1
1.0%
9845
1
1.0%
10423
1
1.0%
12002
1
1.0%
12199
1
1.0%
12356
1
1.0%
13854
1
1.0%
LARGEST VALUES
1555725
1
1.0%
1442727
1
1.0%
1367337
1
1.0%
1330370
1
1.0%
1298287
1
1.0%
1122213
1
1.0%
655339
1
1.0%
564162
1
1.0%
335517
1
1.0%
301172
1
1.0%
283134
1
1.0%
272288
1
1.0%
211873
1
1.0%
211336
1
1.0%
202075
1
1.0%
resultSize
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleBytesIORatio
0.85
numTasks
0.80
executorDeserializeCpuTime
0.56
peakExecutionMemory_GB
0.47
peakExecutionMemory
0.47
shuffleFetchWaitTimeRatio
0.47
resultSerializationTime
0.44
executorDeserializeTime
0.40
shuffleFetchWaitTime
0.12
shuffleLocalBlocksFetched
0.11
submissionTime
0.11
shuffleTotalBlocksFetched
0.08
shuffleRemoteBlocksFetched
0.08
numStages
0.07

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AllSuccessful
0.09
aLogSuccess
0.08
FailureCounts
0.02
AppSuccessful
0.02
diskBytesSpilled
0.02
memoryBytesSpilled
0.02
memoryBytesSpilledRatio
0.02
diskBytesSpilledRatio
0.02
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
4571058
40
38.8%
4571101
9
8.7%
12558115
2
1.9%
24488036
2
1.9%
6388990
1
1.0%
4865173
1
1.0%
5327284
1
1.0%
24201674
1
1.0%
5453359
1
1.0%
28665518
1
1.0%
7157408
1
1.0%
10679711
1
1.0%
8143641
1
1.0%
6867479
1
1.0%
7110422
1
1.0%
SMALLEST VALUES
4571058
40
38.8%
4571101
9
8.7%
4571402
1
1.0%
4747055
1
1.0%
4865173
1
1.0%
4921532
1
1.0%
4922132
1
1.0%
4959438
1
1.0%
5020471
1
1.0%
5327284
1
1.0%
5327413
1
1.0%
5453359
1
1.0%
5526485
1
1.0%
6090899
1
1.0%
6115363
1
1.0%
LARGEST VALUES
816546314
1
1.0%
32332386
1
1.0%
29089409
1
1.0%
28665518
1
1.0%
24488079
1
1.0%
24488036
2
1.9%
24201674
1
1.0%
23303309
1
1.0%
23195979
1
1.0%
23195850
1
1.0%
14580455
1
1.0%
14115446
1
1.0%
13332153
1
1.0%
12949142
1
1.0%
12558115
2
1.9%
diskBytesSpilled
MISSING:
---
TOP CATEGORIES
0
100
97%
89210717932
1
1%
4920412
1
1%
147143467137
1
1%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
diskBytesSpilled
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
memoryBytesSpilled
1.00
shuffleRemoteBytesReadToDisk
1.00
diskBytesSpilledRatio
1.00
memoryBytesSpilledRatio
1.00
FailureCounts
0.20
aLogSuccess
0.03
AllSuccessful
0.03
AppSuccessful
0.01

THESE FEATURES
GIVE INFORMATION
ON diskBytesSpilled:
memoryBytesSpilled
1.00
memoryBytesSpilledRatio
1.00
diskBytesSpilledRatio
0.67
FailureCounts
0.22
aLogSuccess
0.12
AllSuccessful
0.11
AppSuccessful
0.00
App Name
-0.00
RunType
-0.00
shuffleRemoteBytesReadToDisk
-0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
CORRELATION RATIO WITH...
shuffleFetchWaitTime
0.89
aQueryTime
0.87
elapsedTime
0.86
elapsedTime_minutes
0.86
AppDurationSecs
0.85
shuffleRecordsRead
0.78
executorCpuTime
0.65
shuffleRecordsWritten
0.56
jvmGCTime
0.54
stageDuration
0.53
shuffleFetchWaitTimeRatio
0.51
peakExecutionMemory
0.51
peakExecutionMemory_GB
0.51
cpuTimeRatio
0.50
memoryBytesSpilled
MISSING:
---
TOP CATEGORIES
0
100
97%
552406614016
1
1%
434535333888
1
1%
1207959552
1
1%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
memoryBytesSpilled
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
diskBytesSpilled
1.00
shuffleRemoteBytesReadToDisk
1.00
diskBytesSpilledRatio
1.00
memoryBytesSpilledRatio
1.00
FailureCounts
0.20
aLogSuccess
0.03
AllSuccessful
0.03
AppSuccessful
0.01

THESE FEATURES
GIVE INFORMATION
ON memoryBytesSpilled:
diskBytesSpilled
1.00
memoryBytesSpilledRatio
1.00
diskBytesSpilledRatio
0.67
FailureCounts
0.22
aLogSuccess
0.12
AllSuccessful
0.11
AppSuccessful
0.00
App Name
-0.00
RunType
-0.00
shuffleRemoteBytesReadToDisk
-0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
memoryBytesSpilled
CORRELATION RATIO WITH...
shuffleFetchWaitTime
0.89
aQueryTime
0.87
elapsedTime
0.86
elapsedTime_minutes
0.86
AppDurationSecs
0.85
shuffleRecordsRead
0.78
executorCpuTime
0.65
shuffleRecordsWritten
0.56
jvmGCTime
0.54
stageDuration
0.53
shuffleFetchWaitTimeRatio
0.51
peakExecutionMemory
0.51
peakExecutionMemory_GB
0.51
cpuTimeRatio
0.50
peakExecutionMemory
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
peakExecutionMemory_GB
1.00
shuffleRecordsWritten
0.76
cpuTimeRatio
0.74
ioTimeRatio
-0.74
shuffleRemoteBytesRead
0.73
shuffleTotalBytesRead
0.73
shuffleTotalBytesRead_GB
0.73
shuffleRecordsRead
0.73
shuffleLocalBytesRead
0.73
shuffleBytesWritten_GB
0.71
shuffleBytesWritten
0.71
totalBytesWritten_GB
0.71
totalBytesWritten
0.71
executorCpuTime
0.70

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.51
memoryBytesSpilled
0.51
memoryBytesSpilledRatio
0.51
diskBytesSpilledRatio
0.42
AllSuccessful
0.26
aLogSuccess
0.25
FailureCounts
0.12
AppSuccessful
0.08
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
26
25.2%
13474201600
3
2.9%
80845209600
2
1.9%
704907616000
2
1.9%
26869760000
2
1.9%
13407092736
2
1.9%
41379251200
1
1.0%
43686297600
1
1.0%
244947353600
1
1.0%
43620761600
1
1.0%
80039903232
1
1.0%
86099099648
1
1.0%
375809638400
1
1.0%
47815065600
1
1.0%
75485911328
1
1.0%
SMALLEST VALUES
0
26
25.2%
204270704
1
1.0%
332568096
1
1.0%
10320084992
1
1.0%
11795169280
1
1.0%
13004439552
1
1.0%
13407092736
2
1.9%
13474201600
3
2.9%
20198195200
1
1.0%
24904976864
1
1.0%
26869760000
2
1.9%
27250917376
1
1.0%
28534374400
1
1.0%
28882712144
1
1.0%
29351404288
1
1.0%
LARGEST VALUES
981414707200
1
1.0%
850448798976
1
1.0%
752038713600
1
1.0%
704907616000
2
1.9%
638961844224
1
1.0%
590558003200
1
1.0%
468030324736
1
1.0%
375809638400
1
1.0%
325477990400
1
1.0%
302621333432
1
1.0%
254578923776
1
1.0%
244947353600
1
1.0%
237792706368
1
1.0%
213563740320
1
1.0%
191263182400
1
1.0%
recordsRead
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
bytesRead
0.89
totalBytesIORatio
0.81
stageDuration
0.76
executorRunTime
0.66
numStages
0.51
totalBytesRead_GB
0.49
totalBytesRead
0.49
numTasks
0.37
executorCpuTime
0.21
elapsedTime
0.16
elapsedTime_minutes
0.16
jvmGCRatio
-0.16
shuffleLocalBlocksFetched
0.16
AppDurationSecs
0.16

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AllSuccessful
0.16
aLogSuccess
0.15
diskBytesSpilled
0.15
memoryBytesSpilled
0.15
memoryBytesSpilledRatio
0.15
diskBytesSpilledRatio
0.12
AppSuccessful
0.04
FailureCounts
0.03
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
1659654255
3
2.9%
32186120
2
1.9%
4182849091
2
1.9%
9549638501
2
1.9%
3264720721
1
1.0%
720000
1
1.0%
267222581
1
1.0%
78781110
1
1.0%
17278509106
1
1.0%
2093474209
1
1.0%
1323664431
1
1.0%
841070121
1
1.0%
2519004968
1
1.0%
146232849
1
1.0%
SMALLEST VALUES
0
25
24.3%
720000
1
1.0%
20957605
1
1.0%
32186120
2
1.9%
36073071
1
1.0%
41264652
1
1.0%
52797529
1
1.0%
78781110
1
1.0%
133193857
1
1.0%
142579421
1
1.0%
146232849
1
1.0%
160118789
1
1.0%
206353071
1
1.0%
232577449
1
1.0%
264472644
1
1.0%
LARGEST VALUES
129598666156
1
1.0%
69119988350
1
1.0%
51839466426
1
1.0%
23695607901
1
1.0%
17634024863
1
1.0%
17278509106
1
1.0%
14709783488
1
1.0%
14709345129
1
1.0%
9549638501
2
1.9%
9503917218
1
1.0%
8973109521
1
1.0%
8675984124
1
1.0%
7375194761
1
1.0%
5892822403
1
1.0%
4465198630
1
1.0%
bytesRead
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
recordsRead
0.89
executorRunTime
0.79
stageDuration
0.74
totalBytesRead_GB
0.71
totalBytesRead
0.71
totalBytesIORatio
0.61
numStages
0.59
shuffleLocalBlocksFetched
0.44
shuffleTotalBlocksFetched
0.43
shuffleRemoteBlocksFetched
0.43
numTasks
0.40
executorCpuTime
0.39
shuffleWriteTime
0.36
jvmGCTime
0.35

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AllSuccessful
0.26
aLogSuccess
0.24
diskBytesSpilled
0.15
memoryBytesSpilled
0.15
memoryBytesSpilledRatio
0.15
diskBytesSpilledRatio
0.13
AppSuccessful
0.06
FailureCounts
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
108100846171
2
1.9%
169330039
2
1.9%
5904205752
1
1.0%
34822418344
1
1.0%
20844565540
1
1.0%
14060365990
1
1.0%
14410315944
1
1.0%
657360201
1
1.0%
21605125944
1
1.0%
220126580343
1
1.0%
2125370401
1
1.0%
60438137521
1
1.0%
2496611874
1
1.0%
6080329263
1
1.0%
SMALLEST VALUES
0
25
24.3%
5065325
1
1.0%
169330039
2
1.9%
192100734
1
1.0%
192507387
1
1.0%
359997176
1
1.0%
553314458
1
1.0%
657360201
1
1.0%
833434435
1
1.0%
1092247521
1
1.0%
1116953700
1
1.0%
1328162423
1
1.0%
1350591759
1
1.0%
1459938561
1
1.0%
1938557432
1
1.0%
LARGEST VALUES
340144797691
1
1.0%
302385933186
1
1.0%
220126580343
1
1.0%
133103854408
1
1.0%
128532791263
1
1.0%
118909351856
1
1.0%
117411822622
1
1.0%
108100846171
2
1.9%
99829155686
1
1.0%
94557412352
1
1.0%
92951291183
1
1.0%
79862464700
1
1.0%
60438137521
1
1.0%
47109413005
1
1.0%
34822418344
1
1.0%
recordsWritten
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
bytesWritten
0.67
sparkSetupTime
0.29
resultSerializationTime
0.17
shuffleBytesIORatio
0.12
submissionTime
0.11
executorDeserializeTime
0.10
executorDeserializeCpuTime
0.10
totalBytesWritten
0.08
totalBytesWritten_GB
0.08
shuffleBytesWritten_GB
0.08
shuffleBytesWritten
0.08
jvmGCRatio
-0.08
shuffleWriteTime
0.06
shuffleRemoteBlocksFetched
0.05

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
0.09
AllSuccessful
0.07
FailureCounts
0.04
diskBytesSpilled
0.03
memoryBytesSpilled
0.03
memoryBytesSpilledRatio
0.03
AppSuccessful
0.02
diskBytesSpilledRatio
0.02
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
100
53
51.5%
0
27
26.2%
1
5
4.9%
41
1
1.0%
63490
1
1.0%
10
1
1.0%
12
1
1.0%
269
1
1.0%
22
1
1.0%
27
1
1.0%
933
1
1.0%
602
1
1.0%
8366
1
1.0%
48
1
1.0%
272179
1
1.0%
SMALLEST VALUES
0
27
26.2%
1
5
4.9%
10
1
1.0%
12
1
1.0%
22
1
1.0%
27
1
1.0%
41
1
1.0%
48
1
1.0%
52
1
1.0%
76
1
1.0%
100
53
51.5%
269
1
1.0%
602
1
1.0%
858
1
1.0%
933
1
1.0%
LARGEST VALUES
272179
1
1.0%
63490
1
1.0%
53802
1
1.0%
19645
1
1.0%
12383
1
1.0%
8366
1
1.0%
933
1
1.0%
858
1
1.0%
602
1
1.0%
269
1
1.0%
100
53
51.5%
76
1
1.0%
52
1
1.0%
48
1
1.0%
41
1
1.0%
bytesWritten
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
recordsWritten
0.67
sparkSetupTime
0.66
totalBytesWritten
0.27
totalBytesWritten_GB
0.27
shuffleBytesWritten
0.27
shuffleBytesWritten_GB
0.27
shuffleWriteTime
0.23
numStages
0.20
shuffleLocalBytesRead
0.19
shuffleTotalBytesRead
0.18
shuffleTotalBytesRead_GB
0.18
shuffleRemoteBytesRead
0.18
shuffleRemoteBlocksFetched
0.18
shuffleTotalBlocksFetched
0.18

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
0.13
FailureCounts
0.10
AllSuccessful
0.08
diskBytesSpilled
0.04
memoryBytesSpilled
0.04
memoryBytesSpilledRatio
0.04
AppSuccessful
0.03
diskBytesSpilledRatio
0.03
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
26
25.2%
1054
2
1.9%
3382637
1
1.0%
1961
1
1.0%
4267
1
1.0%
4396
1
1.0%
4013
1
1.0%
1583
1
1.0%
71088
1
1.0%
6388
1
1.0%
3394
1
1.0%
13364
1
1.0%
7095
1
1.0%
565560
1
1.0%
4154
1
1.0%
SMALLEST VALUES
0
26
25.2%
571
1
1.0%
779
1
1.0%
1005
1
1.0%
1025
1
1.0%
1054
2
1.9%
1134
1
1.0%
1415
1
1.0%
1538
1
1.0%
1583
1
1.0%
1782
1
1.0%
1784
1
1.0%
1799
1
1.0%
1961
1
1.0%
2072
1
1.0%
LARGEST VALUES
5271035
1
1.0%
4445818
1
1.0%
3382637
1
1.0%
1204065
1
1.0%
574613
1
1.0%
565560
1
1.0%
445653
1
1.0%
430483
1
1.0%
346364
1
1.0%
249158
1
1.0%
71088
1
1.0%
19904
1
1.0%
14681
1
1.0%
13364
1
1.0%
12650
1
1.0%
shuffleRecordsRead
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleTotalBytesRead
0.91
shuffleTotalBytesRead_GB
0.91
shuffleRemoteBytesRead
0.91
shuffleLocalBytesRead
0.91
shuffleRecordsWritten
0.86
shuffleFetchWaitTime
0.86
executorCpuTime
0.85
totalBytesRead
0.82
totalBytesRead_GB
0.82
shuffleWriteTime
0.77
shuffleBytesWritten
0.77
shuffleBytesWritten_GB
0.77
totalBytesWritten
0.77
totalBytesWritten_GB
0.77

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.78
memoryBytesSpilled
0.78
memoryBytesSpilledRatio
0.78
diskBytesSpilledRatio
0.77
AllSuccessful
0.16
aLogSuccess
0.15
FailureCounts
0.06
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
4398222387
2
1.9%
122092207
1
1.0%
2796879904
1
1.0%
8595330723
1
1.0%
862500421
1
1.0%
60164008
1
1.0%
30758432
1
1.0%
39137424169
1
1.0%
7106013189
1
1.0%
696368
1
1.0%
3785265
1
1.0%
2278784
1
1.0%
6278841522
1
1.0%
58589
1
1.0%
SMALLEST VALUES
0
25
24.3%
964
1
1.0%
2319
1
1.0%
9416
1
1.0%
17655
1
1.0%
22066
1
1.0%
58589
1
1.0%
78728
1
1.0%
83222
1
1.0%
86615
1
1.0%
88251
1
1.0%
120639
1
1.0%
129503
1
1.0%
146669
1
1.0%
223885
1
1.0%
LARGEST VALUES
39137424169
1
1.0%
18113255309
1
1.0%
18109707493
1
1.0%
10993651672
1
1.0%
9485557568
1
1.0%
8595330723
1
1.0%
7106013189
1
1.0%
6278841522
1
1.0%
4398222387
2
1.9%
4192235620
1
1.0%
3438611232
1
1.0%
3277470334
1
1.0%
2796879904
1
1.0%
2513145212
1
1.0%
2321855870
1
1.0%
shuffleTotalBlocksFetched
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleRemoteBlocksFetched
1.00
shuffleLocalBlocksFetched
1.00
totalBytesRead_GB
0.72
totalBytesRead
0.72
shuffleWriteTime
0.71
shuffleBytesWritten_GB
0.70
totalBytesWritten_GB
0.70
shuffleBytesWritten
0.70
totalBytesWritten
0.70
executorRunTime
0.70
shuffleTotalBytesRead_GB
0.70
shuffleTotalBytesRead
0.70
shuffleRemoteBytesRead
0.70
shuffleLocalBytesRead
0.70

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.38
memoryBytesSpilled
0.38
memoryBytesSpilledRatio
0.38
diskBytesSpilledRatio
0.37
AllSuccessful
0.31
aLogSuccess
0.30
AppSuccessful
0.08
FailureCounts
0.07
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
813643
2
1.9%
45068
2
1.9%
22966
1
1.0%
147880
1
1.0%
156320
1
1.0%
1408801
1
1.0%
1993634
1
1.0%
86180
1
1.0%
140869
1
1.0%
141607
1
1.0%
440202
1
1.0%
76085
1
1.0%
145950
1
1.0%
878090
1
1.0%
SMALLEST VALUES
0
25
24.3%
452
1
1.0%
2000
1
1.0%
2319
1
1.0%
2796
1
1.0%
4217
1
1.0%
9416
1
1.0%
12400
1
1.0%
15251
1
1.0%
17655
1
1.0%
21065
1
1.0%
22966
1
1.0%
24000
1
1.0%
33563
1
1.0%
34933
1
1.0%
LARGEST VALUES
2566226
1
1.0%
1993634
1
1.0%
1968689
1
1.0%
1408801
1
1.0%
1228442
1
1.0%
1227013
1
1.0%
1209243
1
1.0%
1195816
1
1.0%
896051
1
1.0%
878090
1
1.0%
813643
2
1.9%
771466
1
1.0%
592511
1
1.0%
551417
1
1.0%
550158
1
1.0%
shuffleLocalBlocksFetched
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleTotalBlocksFetched
1.00
shuffleRemoteBlocksFetched
1.00
totalBytesRead_GB
0.72
totalBytesRead
0.72
shuffleWriteTime
0.71
shuffleBytesWritten_GB
0.70
totalBytesWritten_GB
0.70
shuffleBytesWritten
0.70
totalBytesWritten
0.70
executorRunTime
0.70
shuffleRecordsWritten
0.69
shuffleLocalBytesRead
0.69
executorCpuTime
0.69
shuffleTotalBytesRead_GB
0.69

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.38
memoryBytesSpilled
0.38
memoryBytesSpilledRatio
0.38
diskBytesSpilledRatio
0.36
AllSuccessful
0.32
aLogSuccess
0.31
AppSuccessful
0.08
FailureCounts
0.07
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
15
2
1.9%
103
1
1.0%
4639
1
1.0%
416
1
1.0%
911
1
1.0%
419
1
1.0%
804
1
1.0%
1062
1
1.0%
4310
1
1.0%
1837
1
1.0%
4017
1
1.0%
1583
1
1.0%
1187
1
1.0%
817
1
1.0%
SMALLEST VALUES
0
25
24.3%
13
1
1.0%
15
2
1.9%
35
1
1.0%
70
1
1.0%
74
1
1.0%
103
1
1.0%
147
1
1.0%
149
1
1.0%
180
1
1.0%
188
1
1.0%
315
1
1.0%
350
1
1.0%
365
1
1.0%
416
1
1.0%
LARGEST VALUES
20118
1
1.0%
15777
1
1.0%
15496
1
1.0%
11228
1
1.0%
9672
1
1.0%
9619
1
1.0%
9539
1
1.0%
9363
1
1.0%
7038
1
1.0%
6862
1
1.0%
6837
1
1.0%
6397
1
1.0%
6032
1
1.0%
5496
1
1.0%
4639
1
1.0%
shuffleRemoteBlocksFetched
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleTotalBlocksFetched
1.00
shuffleLocalBlocksFetched
1.00
totalBytesRead_GB
0.72
totalBytesRead
0.72
shuffleWriteTime
0.71
shuffleBytesWritten_GB
0.70
totalBytesWritten_GB
0.70
shuffleBytesWritten
0.70
totalBytesWritten
0.70
executorRunTime
0.70
shuffleTotalBytesRead_GB
0.70
shuffleTotalBytesRead
0.70
shuffleRemoteBytesRead
0.70
shuffleLocalBytesRead
0.70

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.38
memoryBytesSpilled
0.38
memoryBytesSpilledRatio
0.38
diskBytesSpilledRatio
0.37
AllSuccessful
0.31
aLogSuccess
0.30
AppSuccessful
0.08
FailureCounts
0.07
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
337189
1
1.0%
547107
1
1.0%
323230
1
1.0%
44703
1
1.0%
142112
1
1.0%
55073
1
1.0%
146722
1
1.0%
143267
1
1.0%
204967
1
1.0%
258842
1
1.0%
436776
1
1.0%
1953193
1
1.0%
144811
1
1.0%
44718
1
1.0%
SMALLEST VALUES
0
25
24.3%
437
1
1.0%
1985
1
1.0%
2306
1
1.0%
2722
1
1.0%
4182
1
1.0%
9346
1
1.0%
12297
1
1.0%
13501
1
1.0%
17506
1
1.0%
20918
1
1.0%
22786
1
1.0%
23812
1
1.0%
31726
1
1.0%
34618
1
1.0%
LARGEST VALUES
2546108
1
1.0%
1977857
1
1.0%
1953193
1
1.0%
1397573
1
1.0%
1218823
1
1.0%
1217474
1
1.0%
1199571
1
1.0%
1186453
1
1.0%
889013
1
1.0%
871228
1
1.0%
807246
1
1.0%
806806
1
1.0%
765434
1
1.0%
587872
1
1.0%
547107
1
1.0%
shuffleTotalBytesRead
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleTotalBytesRead_GB
1.00
shuffleRemoteBytesRead
1.00
shuffleLocalBytesRead
1.00
shuffleBytesWritten_GB
0.93
shuffleBytesWritten
0.93
totalBytesWritten_GB
0.93
totalBytesWritten
0.93
shuffleRecordsRead
0.91
shuffleWriteTime
0.91
totalBytesRead
0.90
totalBytesRead_GB
0.90
shuffleRecordsWritten
0.85
executorCpuTime
0.80
peakExecutionMemory
0.73

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.49
memoryBytesSpilled
0.49
memoryBytesSpilledRatio
0.49
diskBytesSpilledRatio
0.49
AllSuccessful
0.19
aLogSuccess
0.18
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
4375294
1
1.0%
10252674
1
1.0%
2201092389
1
1.0%
4150289319
1
1.0%
1263948072
1
1.0%
44160631724
1
1.0%
48410923
1
1.0%
4370476
1
1.0%
42102256
1
1.0%
966906145
1
1.0%
551745840
1
1.0%
16589023536
1
1.0%
45599990048
1
1.0%
155874364198
1
1.0%
SMALLEST VALUES
0
25
24.3%
51667
1
1.0%
136600
1
1.0%
555395
1
1.0%
1116918
1
1.0%
4370476
1
1.0%
4375294
1
1.0%
5830177
1
1.0%
6143287
1
1.0%
6518977
1
1.0%
6692175
1
1.0%
6719804
1
1.0%
8997961
1
1.0%
10252674
1
1.0%
14797454
1
1.0%
LARGEST VALUES
482205891654
1
1.0%
481957465913
1
1.0%
476617967742
1
1.0%
332346489177
1
1.0%
247489323749
1
1.0%
177856880630
1
1.0%
155874364198
1
1.0%
99724873661
1
1.0%
85533271489
1
1.0%
65533355045
1
1.0%
61850513462
1
1.0%
46568674778
1
1.0%
45599990048
1
1.0%
45385550907
1
1.0%
44160631724
1
1.0%
shuffleLocalBytesRead
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleTotalBytesRead_GB
1.00
shuffleTotalBytesRead
1.00
shuffleRemoteBytesRead
1.00
shuffleBytesWritten_GB
0.93
shuffleBytesWritten
0.93
totalBytesWritten_GB
0.93
totalBytesWritten
0.93
shuffleRecordsRead
0.91
shuffleWriteTime
0.91
totalBytesRead
0.90
totalBytesRead_GB
0.90
shuffleRecordsWritten
0.86
executorCpuTime
0.80
peakExecutionMemory
0.73

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.49
memoryBytesSpilled
0.49
memoryBytesSpilledRatio
0.49
diskBytesSpilledRatio
0.49
AllSuccessful
0.20
aLogSuccess
0.19
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
17143986
1
1.0%
35242
1
1.0%
157648171
1
1.0%
361349164
1
1.0%
68323758
1
1.0%
24684591
1
1.0%
109598170
1
1.0%
258865
1
1.0%
621716856
1
1.0%
9857070
1
1.0%
198586081
1
1.0%
3783179789
1
1.0%
19609527
1
1.0%
10962745
1
1.0%
SMALLEST VALUES
0
25
24.3%
766
1
1.0%
1651
1
1.0%
4130
1
1.0%
9432
1
1.0%
35242
1
1.0%
35565
1
1.0%
45648
1
1.0%
45947
1
1.0%
51548
1
1.0%
51771
1
1.0%
53843
1
1.0%
67318
1
1.0%
82050
1
1.0%
116714
1
1.0%
LARGEST VALUES
3783179789
1
1.0%
3761619415
1
1.0%
3742840935
1
1.0%
2659490450
1
1.0%
1933802884
1
1.0%
1392739553
1
1.0%
1235775345
1
1.0%
779643737
1
1.0%
621716856
1
1.0%
530326865
1
1.0%
484375012
1
1.0%
409006875
1
1.0%
361349164
1
1.0%
352791109
1
1.0%
345129799
1
1.0%
shuffleRemoteBytesRead
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleTotalBytesRead
1.00
shuffleTotalBytesRead_GB
1.00
shuffleLocalBytesRead
1.00
shuffleBytesWritten_GB
0.93
shuffleBytesWritten
0.93
totalBytesWritten_GB
0.93
totalBytesWritten
0.93
shuffleRecordsRead
0.91
shuffleWriteTime
0.91
totalBytesRead
0.90
totalBytesRead_GB
0.90
shuffleRecordsWritten
0.85
executorCpuTime
0.80
peakExecutionMemory
0.73

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.49
memoryBytesSpilled
0.49
memoryBytesSpilledRatio
0.49
diskBytesSpilledRatio
0.49
AllSuccessful
0.19
aLogSuccess
0.18
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
46207325614
1
1.0%
472875126807
1
1.0%
10170624
1
1.0%
524760180
1
1.0%
14680740
1
1.0%
35231781
1
1.0%
154638588853
1
1.0%
1093741096
1
1.0%
42786106
1
1.0%
135834
1
1.0%
48009524
1
1.0%
542155957
1
1.0%
8712455904
1
1.0%
9431184861
1
1.0%
SMALLEST VALUES
0
25
24.3%
50016
1
1.0%
135834
1
1.0%
551265
1
1.0%
1107486
1
1.0%
4335234
1
1.0%
4339729
1
1.0%
5784529
1
1.0%
6091739
1
1.0%
6467206
1
1.0%
6646228
1
1.0%
6665961
1
1.0%
8930643
1
1.0%
10170624
1
1.0%
14680740
1
1.0%
LARGEST VALUES
478444272239
1
1.0%
478174286124
1
1.0%
472875126807
1
1.0%
329686998727
1
1.0%
245555520865
1
1.0%
176464141077
1
1.0%
154638588853
1
1.0%
98945229924
1
1.0%
84911554633
1
1.0%
65003028180
1
1.0%
61366138450
1
1.0%
46207325614
1
1.0%
45190983173
1
1.0%
45032759798
1
1.0%
43815501925
1
1.0%
shuffleRemoteBytesReadToDisk
MISSING:
---
TOP CATEGORIES
0
103
100%
ALL
103
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
shuffleRemoteBytesReadToDisk
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
aLogSuccess
0.00
AllSuccessful
0.00
diskBytesSpilled
-0.00
memoryBytesSpilled
-0.00
memoryBytesSpilledRatio
-0.00
diskBytesSpilledRatio
-0.00
AppSuccessful
-0.00
FailureCounts
-0.00

THESE FEATURES
GIVE INFORMATION
ON shuffleRemoteBytesReadToDisk:
App Name
1.00
AllSuccessful
1.00
aLogSuccess
1.00
AppSuccessful
1.00
FailureCounts
1.00
RunType
1.00
diskBytesSpilled
1.00
memoryBytesSpilled
1.00
diskBytesSpilledRatio
1.00
memoryBytesSpilledRatio
1.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
shuffleRemoteBytesReadToDisk
CORRELATION RATIO WITH...
aQueryTime
0.00
elapsedTime
0.00
AppDurationSecs
0.00
stageDuration
0.00
numStages
0.00
numTasks
0.00
executorRunTime
0.00
executorCpuTime
0.00
executorDeserializeTime
0.00
executorDeserializeCpuTime
0.00
resultSerializationTime
0.00
jvmGCTime
0.00
shuffleFetchWaitTime
0.00
shuffleWriteTime
0.00
shuffleBytesWritten
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleBytesWritten_GB
1.00
totalBytesWritten_GB
1.00
totalBytesWritten
1.00
shuffleWriteTime
0.97
shuffleLocalBytesRead
0.93
shuffleTotalBytesRead
0.93
shuffleTotalBytesRead_GB
0.93
shuffleRemoteBytesRead
0.93
shuffleRecordsWritten
0.87
totalBytesRead_GB
0.85
totalBytesRead
0.85
executorCpuTime
0.79
cpuTimeRatio
0.77
ioTimeRatio
-0.77

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.29
memoryBytesSpilled
0.29
memoryBytesSpilledRatio
0.29
diskBytesSpilledRatio
0.29
AllSuccessful
0.22
aLogSuccess
0.21
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
42102256
1
1.0%
5038407842
1
1.0%
10252674
1
1.0%
2201092389
1
1.0%
4150289319
1
1.0%
18568689
1
1.0%
177856880630
1
1.0%
4370476
1
1.0%
14797454
1
1.0%
247304516370
1
1.0%
16589023536
1
1.0%
1000398910
1
1.0%
3562338259
1
1.0%
6143287
1
1.0%
SMALLEST VALUES
0
25
24.3%
51667
1
1.0%
136600
1
1.0%
555395
1
1.0%
1116918
1
1.0%
4370476
1
1.0%
4375294
1
1.0%
5830177
1
1.0%
6143287
1
1.0%
6518977
1
1.0%
6692175
1
1.0%
6719804
1
1.0%
7155271
1
1.0%
8997961
1
1.0%
10252674
1
1.0%
LARGEST VALUES
318522671539
1
1.0%
247489323749
1
1.0%
247304516370
1
1.0%
247056090629
1
1.0%
180699300708
1
1.0%
177856880630
1
1.0%
144558172958
1
1.0%
85533271489
1
1.0%
66214058527
1
1.0%
61850513462
1
1.0%
58743640301
1
1.0%
46568674778
1
1.0%
45385550907
1
1.0%
44160631724
1
1.0%
42649310711
1
1.0%
shuffleRecordsWritten
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleBytesWritten
0.87
shuffleBytesWritten_GB
0.87
totalBytesWritten
0.87
totalBytesWritten_GB
0.87
shuffleRecordsRead
0.86
shuffleLocalBytesRead
0.86
shuffleWriteTime
0.85
shuffleTotalBytesRead
0.85
shuffleTotalBytesRead_GB
0.85
shuffleRemoteBytesRead
0.85
executorCpuTime
0.84
cpuTimeRatio
0.81
ioTimeRatio
-0.81
totalBytesRead_GB
0.78

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.56
memoryBytesSpilled
0.56
memoryBytesSpilledRatio
0.56
diskBytesSpilledRatio
0.48
AllSuccessful
0.20
aLogSuccess
0.19
FailureCounts
0.12
AppSuccessful
0.06
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
4308222387
2
1.9%
122092207
1
1.0%
148386593
1
1.0%
129503
1
1.0%
846062191
1
1.0%
862500421
1
1.0%
83222
1
1.0%
9145451105
1
1.0%
7106013189
1
1.0%
696368
1
1.0%
605758111
1
1.0%
2278784
1
1.0%
6128841522
1
1.0%
58589
1
1.0%
SMALLEST VALUES
0
25
24.3%
964
1
1.0%
2319
1
1.0%
9416
1
1.0%
17655
1
1.0%
22066
1
1.0%
58589
1
1.0%
78728
1
1.0%
83222
1
1.0%
86615
1
1.0%
88251
1
1.0%
120639
1
1.0%
129503
1
1.0%
146669
1
1.0%
223885
1
1.0%
LARGEST VALUES
14264115130
1
1.0%
9985778920
1
1.0%
9485557568
1
1.0%
9148998921
1
1.0%
9145451105
1
1.0%
8795004910
1
1.0%
8639913154
1
1.0%
7106013189
1
1.0%
6128841522
1
1.0%
4320005325
1
1.0%
4308222387
2
1.9%
3438611232
1
1.0%
3277470334
1
1.0%
2796879904
1
1.0%
2680223262
1
1.0%
submissionTime
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
jvmGCRatio
-0.26
sparkSetupTime
0.24
resultSerializationTime
-0.20
jvmGCTime
-0.15
bytesWritten
0.15
shuffleFetchWaitTime
-0.13
numStages
-0.11
resultSize
0.11
shuffleRecordsRead
-0.11
recordsWritten
0.11
stageDuration
-0.09
totalBytesRead_GB
-0.08
totalBytesRead
-0.08
recordsRead
-0.08

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.16
memoryBytesSpilled
0.16
memoryBytesSpilledRatio
0.16
FailureCounts
0.12
AppSuccessful
0.11
diskBytesSpilledRatio
0.10
aLogSuccess
0.02
AllSuccessful
0.00
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
1604522458632
1
1.0%
1604517083747
1
1.0%
1604517718260
1
1.0%
1604518251752
1
1.0%
1604515474670
1
1.0%
1604523062255
1
1.0%
1604522886381
1
1.0%
1604519475426
1
1.0%
1604522716131
1
1.0%
1604524406312
1
1.0%
1604519005663
1
1.0%
1604519987466
1
1.0%
1604516697303
1
1.0%
1604522565196
1
1.0%
1604519804628
1
1.0%
SMALLEST VALUES
1604514675204
1
1.0%
1604514733655
1
1.0%
1604514772015
1
1.0%
1604514835618
1
1.0%
1604515082517
1
1.0%
1604515168840
1
1.0%
1604515229909
1
1.0%
1604515284910
1
1.0%
1604515323213
1
1.0%
1604515474670
1
1.0%
1604515540741
1
1.0%
1604515636931
1
1.0%
1604515688617
1
1.0%
1604515727459
1
1.0%
1604515768940
1
1.0%
LARGEST VALUES
1604524893889
1
1.0%
1604524667780
1
1.0%
1604524452878
1
1.0%
1604524406312
1
1.0%
1604524359996
1
1.0%
1604524314621
1
1.0%
1604524103285
1
1.0%
1604524056770
1
1.0%
1604523990943
1
1.0%
1604523921613
1
1.0%
1604523860758
1
1.0%
1604523684414
1
1.0%
1604523609963
1
1.0%
1604523548428
1
1.0%
1604523502375
1
1.0%
cpuTimeRatio
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
ioTimeRatio
-1.00
executorCpuTime
0.86
shuffleRecordsWritten
0.81
shuffleBytesWritten
0.77
shuffleBytesWritten_GB
0.77
totalBytesWritten
0.77
totalBytesWritten_GB
0.77
shuffleWriteTime
0.77
peakExecutionMemory_GB
0.74
peakExecutionMemory
0.74
shuffleLocalBytesRead
0.71
shuffleTotalBytesRead
0.71
shuffleTotalBytesRead_GB
0.71
shuffleRemoteBytesRead
0.71

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.50
memoryBytesSpilled
0.50
memoryBytesSpilledRatio
0.50
diskBytesSpilledRatio
0.43
AllSuccessful
0.31
aLogSuccess
0.30
FailureCounts
0.12
AppSuccessful
0.10
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.062
7
6.8%
0.06
5
4.9%
0.097
5
4.9%
0.088
4
3.9%
0.116
4
3.9%
0.08
3
2.9%
0.099
3
2.9%
0.09
3
2.9%
0.063
3
2.9%
0.091
2
1.9%
0.104
2
1.9%
0.234
2
1.9%
0.05
2
1.9%
0.1
2
1.9%
0.059
2
1.9%
SMALLEST VALUES
0.027
1
1.0%
0.04
2
1.9%
0.042
1
1.0%
0.044
1
1.0%
0.049
1
1.0%
0.05
2
1.9%
0.051
1
1.0%
0.052
1
1.0%
0.053
2
1.9%
0.058
1
1.0%
0.059
2
1.9%
0.06
5
4.9%
0.061
1
1.0%
0.062
7
6.8%
0.063
3
2.9%
LARGEST VALUES
0.636
1
1.0%
0.494
1
1.0%
0.489
1
1.0%
0.485
1
1.0%
0.422
1
1.0%
0.41
1
1.0%
0.408
1
1.0%
0.406
1
1.0%
0.361
1
1.0%
0.338
1
1.0%
0.263
1
1.0%
0.246
1
1.0%
0.244
1
1.0%
0.234
2
1.9%
0.204
1
1.0%
ioTimeRatio
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
cpuTimeRatio
-1.00
executorCpuTime
-0.86
shuffleRecordsWritten
-0.81
shuffleBytesWritten
-0.77
shuffleBytesWritten_GB
-0.77
totalBytesWritten
-0.77
totalBytesWritten_GB
-0.77
shuffleWriteTime
-0.77
peakExecutionMemory_GB
-0.74
peakExecutionMemory
-0.74
shuffleLocalBytesRead
-0.71
shuffleTotalBytesRead
-0.71
shuffleTotalBytesRead_GB
-0.71
shuffleRemoteBytesRead
-0.71

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.50
memoryBytesSpilled
0.50
memoryBytesSpilledRatio
0.50
diskBytesSpilledRatio
0.43
AllSuccessful
0.31
aLogSuccess
0.30
FailureCounts
0.12
AppSuccessful
0.10
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.938
7
6.8%
0.94
5
4.9%
0.903
5
4.9%
0.884
4
3.9%
0.912
4
3.9%
0.901
3
2.9%
0.92
3
2.9%
0.937
3
2.9%
0.91
3
2.9%
0.95
2
1.9%
0.906
2
1.9%
0.766
2
1.9%
0.96
2
1.9%
0.896
2
1.9%
0.947
2
1.9%
SMALLEST VALUES
0.364
1
1.0%
0.506
1
1.0%
0.511
1
1.0%
0.515
1
1.0%
0.578
1
1.0%
0.59
1
1.0%
0.592
1
1.0%
0.594
1
1.0%
0.639
1
1.0%
0.662
1
1.0%
0.737
1
1.0%
0.754
1
1.0%
0.756
1
1.0%
0.766
2
1.9%
0.796
1
1.0%
LARGEST VALUES
0.973
1
1.0%
0.96
2
1.9%
0.958
1
1.0%
0.956
1
1.0%
0.951
1
1.0%
0.95
2
1.9%
0.949
1
1.0%
0.948
1
1.0%
0.947
2
1.9%
0.942
1
1.0%
0.941
2
1.9%
0.94
5
4.9%
0.939
1
1.0%
0.938
7
6.8%
0.937
3
2.9%
jvmGCRatio
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
jvmGCTime
0.67
shuffleFetchWaitTimeRatio
0.42
shuffleFetchWaitTime
0.40
cpuTimeRatio
0.38
ioTimeRatio
-0.38
executorCpuTime
0.37
shuffleWriteTime
0.37
shuffleRemoteBlocksFetched
0.35
shuffleTotalBlocksFetched
0.35
shuffleLocalBlocksFetched
0.35
executorDeserializeTime
0.34
shuffleRecordsWritten
0.30
totalBytesIORatio
-0.30
shuffleRecordsRead
0.28

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.31
memoryBytesSpilled
0.31
memoryBytesSpilledRatio
0.31
diskBytesSpilledRatio
0.31
AllSuccessful
0.23
aLogSuccess
0.23
AppSuccessful
0.08
FailureCounts
0.03
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.055
5
4.9%
0.067
4
3.9%
0.056
4
3.9%
0.047
4
3.9%
0.058
4
3.9%
0.06
4
3.9%
0.053
3
2.9%
0.049
3
2.9%
0.069
3
2.9%
0.051
3
2.9%
0.066
3
2.9%
0.078
3
2.9%
0.057
3
2.9%
0.059
3
2.9%
0.063
2
1.9%
SMALLEST VALUES
0.013
1
1.0%
0.014
1
1.0%
0.025
1
1.0%
0.027
1
1.0%
0.029
1
1.0%
0.033
1
1.0%
0.036
1
1.0%
0.037
1
1.0%
0.038
2
1.9%
0.04
2
1.9%
0.041
2
1.9%
0.042
2
1.9%
0.043
1
1.0%
0.044
1
1.0%
0.045
1
1.0%
LARGEST VALUES
0.184
1
1.0%
0.175
1
1.0%
0.172
1
1.0%
0.131
1
1.0%
0.13
1
1.0%
0.128
1
1.0%
0.123
2
1.9%
0.111
1
1.0%
0.099
1
1.0%
0.094
1
1.0%
0.09
1
1.0%
0.087
2
1.9%
0.086
1
1.0%
0.084
1
1.0%
0.081
2
1.9%
shuffleFetchWaitTimeRatio
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleFetchWaitTime
0.76
numTasks
0.63
peakExecutionMemory
0.61
peakExecutionMemory_GB
0.61
shuffleRecordsRead
0.59
shuffleBytesIORatio
0.57
shuffleLocalBlocksFetched
0.55
shuffleTotalBlocksFetched
0.53
shuffleRemoteBlocksFetched
0.53
jvmGCTime
0.53
executorCpuTime
0.52
shuffleRecordsWritten
0.51
numStages
0.50
shuffleLocalBytesRead
0.49

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.51
memoryBytesSpilled
0.51
memoryBytesSpilledRatio
0.51
diskBytesSpilledRatio
0.50
AllSuccessful
0.22
aLogSuccess
0.21
AppSuccessful
0.06
FailureCounts
0.04
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.0
65
63.1%
0.001
10
9.7%
0.002
6
5.8%
0.003
3
2.9%
0.004
2
1.9%
0.017
2
1.9%
0.006
2
1.9%
0.008
2
1.9%
0.031
1
1.0%
0.048
1
1.0%
0.044
1
1.0%
0.011
1
1.0%
0.014
1
1.0%
0.042
1
1.0%
0.015
1
1.0%
SMALLEST VALUES
0.0
65
63.1%
0.001
10
9.7%
0.002
6
5.8%
0.003
3
2.9%
0.004
2
1.9%
0.005
1
1.0%
0.006
2
1.9%
0.008
2
1.9%
0.01
1
1.0%
0.011
1
1.0%
0.014
1
1.0%
0.015
1
1.0%
0.017
2
1.9%
0.018
1
1.0%
0.019
1
1.0%
LARGEST VALUES
0.048
1
1.0%
0.044
1
1.0%
0.042
1
1.0%
0.031
1
1.0%
0.019
1
1.0%
0.018
1
1.0%
0.017
2
1.9%
0.015
1
1.0%
0.014
1
1.0%
0.011
1
1.0%
0.01
1
1.0%
0.008
2
1.9%
0.006
2
1.9%
0.005
1
1.0%
0.004
2
1.9%
totalBytesRead
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
totalBytesRead_GB
1.00
shuffleTotalBytesRead_GB
0.90
shuffleTotalBytesRead
0.90
shuffleRemoteBytesRead
0.90
shuffleLocalBytesRead
0.90
shuffleBytesWritten_GB
0.85
shuffleBytesWritten
0.85
totalBytesWritten_GB
0.85
totalBytesWritten
0.85
shuffleWriteTime
0.84
executorRunTime
0.82
shuffleRecordsRead
0.82
executorCpuTime
0.78
shuffleRecordsWritten
0.78

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.42
memoryBytesSpilled
0.42
memoryBytesSpilledRatio
0.42
diskBytesSpilledRatio
0.42
AllSuccessful
0.26
aLogSuccess
0.25
AppSuccessful
0.06
FailureCounts
0.06
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
15784954031
1
1.0%
20854818214
1
1.0%
2574456741
1
1.0%
576447123428
1
1.0%
87208103714
1
1.0%
3846900065
1
1.0%
12402137005
1
1.0%
14008398892
1
1.0%
340440614932
1
1.0%
73748600493
1
1.0%
2153969077
1
1.0%
64382885681
1
1.0%
11727881398
1
1.0%
7336062460
1
1.0%
SMALLEST VALUES
0
25
24.3%
5116992
1
1.0%
227591380
1
1.0%
456645227
1
1.0%
702261910
1
1.0%
721075879
1
1.0%
853634122
1
1.0%
1541359578
1
1.0%
1608131303
1
1.0%
2129745695
1
1.0%
2153969077
1
1.0%
2173178858
1
1.0%
2574456741
1
1.0%
3034526824
1
1.0%
3105241019
1
1.0%
LARGEST VALUES
590306737825
1
1.0%
590058312084
1
1.0%
576447123428
1
1.0%
451255841033
1
1.0%
340440614932
1
1.0%
340145914609
1
1.0%
303488501636
1
1.0%
250431776550
1
1.0%
224966293635
1
1.0%
220127135738
1
1.0%
179587338361
1
1.0%
133109997695
1
1.0%
128862444950
1
1.0%
117460233545
1
1.0%
98161374243
1
1.0%
totalBytesWritten
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
totalBytesWritten_GB
1.00
shuffleBytesWritten
1.00
shuffleBytesWritten_GB
1.00
shuffleWriteTime
0.97
shuffleLocalBytesRead
0.93
shuffleTotalBytesRead
0.93
shuffleTotalBytesRead_GB
0.93
shuffleRemoteBytesRead
0.93
shuffleRecordsWritten
0.87
totalBytesRead_GB
0.85
totalBytesRead
0.85
executorCpuTime
0.79
cpuTimeRatio
0.77
ioTimeRatio
-0.77

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.29
memoryBytesSpilled
0.29
memoryBytesSpilledRatio
0.29
diskBytesSpilledRatio
0.29
AllSuccessful
0.22
aLogSuccess
0.21
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0
25
24.3%
144558176288
1
1.0%
3562340643
1
1.0%
4372260
1
1.0%
9389526826
1
1.0%
1918114070
1
1.0%
7159058
1
1.0%
264150316
1
1.0%
11868359340
1
1.0%
9518755379
1
1.0%
46568677581
1
1.0%
180699302945
1
1.0%
628351925
1
1.0%
6521904
1
1.0%
1000745274
1
1.0%
SMALLEST VALUES
0
25
24.3%
53250
1
1.0%
137171
1
1.0%
557653
1
1.0%
1118456
1
1.0%
4372260
1
1.0%
4378007
1
1.0%
5833571
1
1.0%
6146386
1
1.0%
6521904
1
1.0%
6712079
1
1.0%
6722672
1
1.0%
7159058
1
1.0%
9000905
1
1.0%
10256521
1
1.0%
LARGEST VALUES
318527117357
1
1.0%
247489324803
1
1.0%
247304962023
1
1.0%
247056339787
1
1.0%
180699302945
1
1.0%
177856887119
1
1.0%
144558176288
1
1.0%
85533275502
1
1.0%
66214062464
1
1.0%
61850525181
1
1.0%
58743643649
1
1.0%
46568677581
1
1.0%
45385563354
1
1.0%
44160635878
1
1.0%
42649313783
1
1.0%
diskBytesSpilledRatio
MISSING:
25
(24%)
TOP CATEGORIES
0.0
76
97%
0.255
1
1%
0.909
1
1%
ALL
78
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
diskBytesSpilledRatio
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
shuffleRemoteBytesReadToDisk
1.00
diskBytesSpilled
0.67
memoryBytesSpilled
0.67
memoryBytesSpilledRatio
0.67
AllSuccessful
0.01
aLogSuccess
0.01
FailureCounts
0.00
AppSuccessful
0.00

THESE FEATURES
GIVE INFORMATION
ON diskBytesSpilledRatio:
diskBytesSpilled
1.00
memoryBytesSpilled
1.00
memoryBytesSpilledRatio
1.00
AllSuccessful
0.06
aLogSuccess
0.05
FailureCounts
0.01
AppSuccessful
0.00
App Name
-0.00
RunType
-0.00
shuffleRemoteBytesReadToDisk
-0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilledRatio
CORRELATION RATIO WITH...
shuffleFetchWaitTime
0.89
aQueryTime
0.87
elapsedTime_minutes
0.82
elapsedTime
0.82
AppDurationSecs
0.82
shuffleRecordsRead
0.77
executorCpuTime
0.63
jvmGCTime
0.54
stageDuration
0.50
shuffleFetchWaitTimeRatio
0.50
shuffleRemoteBytesRead
0.49
shuffleTotalBytesRead_GB
0.49
shuffleTotalBytesRead
0.49
shuffleLocalBytesRead
0.49
memoryBytesSpilledRatio
MISSING:
25
(24%)
TOP CATEGORIES
0.0
75
96%
0.014
1
1%
0.958
1
1%
4.427
1
1%
ALL
78
100%
CATEGORICAL ASSOCIATIONS
(UNCERTAINTY COEFFICIENT, 0 to 1)
memoryBytesSpilledRatio
PROVIDES INFORMATION ON...
App Name
1.00
RunType
1.00
diskBytesSpilled
1.00
memoryBytesSpilled
1.00
shuffleRemoteBytesReadToDisk
1.00
diskBytesSpilledRatio
1.00
FailureCounts
0.20
aLogSuccess
0.03
AllSuccessful
0.03
AppSuccessful
0.01

THESE FEATURES
GIVE INFORMATION
ON memoryBytesSpilledRatio:
diskBytesSpilled
1.00
memoryBytesSpilled
1.00
diskBytesSpilledRatio
0.67
FailureCounts
0.22
aLogSuccess
0.12
AllSuccessful
0.11
AppSuccessful
0.00
App Name
-0.00
RunType
-0.00
shuffleRemoteBytesReadToDisk
-0.00

NUMERICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
memoryBytesSpilledRatio
CORRELATION RATIO WITH...
shuffleFetchWaitTime
0.89
aQueryTime
0.87
elapsedTime
0.86
elapsedTime_minutes
0.86
AppDurationSecs
0.85
shuffleRecordsRead
0.78
executorCpuTime
0.65
shuffleRecordsWritten
0.56
jvmGCTime
0.54
stageDuration
0.53
shuffleFetchWaitTimeRatio
0.51
peakExecutionMemory
0.51
peakExecutionMemory_GB
0.51
cpuTimeRatio
0.50
totalBytesIORatio
MISSING:
25
(24%)
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
recordsRead
0.81
bytesRead
0.61
stageDuration
0.56
executorRunTime
0.45
numStages
0.42
jvmGCRatio
-0.30
resultSerializationTime
-0.25
numTasks
0.24
totalBytesRead
0.22
totalBytesRead_GB
0.22
executorDeserializeCpuTime
-0.14
executorDeserializeTime
-0.14
shuffleLocalBlocksFetched
-0.13
ioTimeRatio
0.13

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AllSuccessful
0.10
aLogSuccess
0.10
FailureCounts
0.03
diskBytesSpilled
0.03
memoryBytesSpilled
0.03
memoryBytesSpilledRatio
0.03
AppSuccessful
0.02
diskBytesSpilledRatio
0.02
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
1.376
2
2.6%
1.729
2
2.6%
1.423
1
1.3%
2.415
1
1.3%
1.607
1
1.3%
88.204
1
1.3%
1.732
1
1.3%
2.387
1
1.3%
275.255
1
1.3%
1.963
1
1.3%
1.478
1
1.3%
1173.286
1
1.3%
394738.548
1
1.3%
1878.3
1
1.3%
1.675
1
1.3%
SMALLEST VALUES
0.932
1
1.3%
1.001
1
1.3%
1.148
1
1.3%
1.265
1
1.3%
1.321
1
1.3%
1.336
1
1.3%
1.342
1
1.3%
1.35
1
1.3%
1.355
1
1.3%
1.376
2
2.6%
1.379
1
1.3%
1.406
1
1.3%
1.415
1
1.3%
1.417
1
1.3%
1.419
1
1.3%
LARGEST VALUES
394738.548
1
1.3%
304120.962
1
1.3%
102123.619
1
1.3%
21656.628
1
1.3%
2544.931
1
1.3%
2348.018
1
1.3%
2033.323
1
1.3%
1893.731
1
1.3%
1878.3
1
1.3%
1173.286
1
1.3%
773.899
1
1.3%
599.999
1
1.3%
599.945
1
1.3%
486.465
1
1.3%
433.75
1
1.3%
shuffleBytesIORatio
MISSING:
25
(24%)
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
resultSize
0.85
numTasks
0.76
executorDeserializeCpuTime
0.76
peakExecutionMemory_GB
0.58
peakExecutionMemory
0.58
shuffleFetchWaitTimeRatio
0.57
resultSerializationTime
0.52
executorDeserializeTime
0.51
shuffleFetchWaitTime
0.35
shuffleRecordsRead
0.27
shuffleLocalBlocksFetched
0.27
shuffleTotalBlocksFetched
0.26
shuffleRemoteBlocksFetched
0.26
shuffleRemoteBytesRead
0.25

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
aLogSuccess
0.64
AllSuccessful
0.59
diskBytesSpilled
0.23
memoryBytesSpilled
0.23
memoryBytesSpilledRatio
0.23
diskBytesSpilledRatio
0.23
AppSuccessful
0.16
FailureCounts
0.16
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
1.0
47
60.3%
1.016
3
3.8%
1.506
1
1.3%
0.142
1
1.3%
1.114
1
1.3%
1.878
1
1.3%
1.018
1
1.3%
1.98
1
1.3%
0.068
1
1.3%
1.02
1
1.3%
1.908
1
1.3%
1.95
1
1.3%
1.158
1
1.3%
1.992
1
1.3%
0.781
1
1.3%
SMALLEST VALUES
0.068
1
1.3%
0.142
1
1.3%
0.781
1
1.3%
0.987
1
1.3%
1.0
47
60.3%
1.001
1
1.3%
1.012
1
1.3%
1.014
1
1.3%
1.016
3
3.8%
1.018
1
1.3%
1.02
1
1.3%
1.027
1
1.3%
1.029
1
1.3%
1.036
1
1.3%
1.043
1
1.3%
LARGEST VALUES
6.039
1
1.3%
2.638
1
1.3%
1.992
1
1.3%
1.98
1
1.3%
1.951
1
1.3%
1.95
1
1.3%
1.908
1
1.3%
1.878
1
1.3%
1.506
1
1.3%
1.172
1
1.3%
1.158
1
1.3%
1.116
1
1.3%
1.114
1
1.3%
1.078
1
1.3%
1.069
1
1.3%
elapsedTime_minutes
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
elapsedTime
1.00
AppDurationSecs
1.00
aQueryTime
0.99
cpuTimeRatio
0.71
ioTimeRatio
-0.71
stageDuration
0.68
executorCpuTime
0.63
peakExecutionMemory
0.56
peakExecutionMemory_GB
0.56
executorRunTime
0.51
shuffleRecordsWritten
0.50
shuffleBytesWritten
0.48
totalBytesWritten
0.48
shuffleBytesWritten_GB
0.48

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.86
memoryBytesSpilled
0.86
memoryBytesSpilledRatio
0.86
diskBytesSpilledRatio
0.82
AllSuccessful
0.21
aLogSuccess
0.20
FailureCounts
0.10
AppSuccessful
0.04
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.554
4
3.9%
0.919
2
1.9%
0.537
2
1.9%
0.56
1
1.0%
0.955
1
1.0%
0.831
1
1.0%
1.338
1
1.0%
0.706
1
1.0%
1.743
1
1.0%
0.906
1
1.0%
1.05
1
1.0%
0.546
1
1.0%
0.882
1
1.0%
1.117
1
1.0%
1.307
1
1.0%
SMALLEST VALUES
0.537
2
1.9%
0.54
1
1.0%
0.542
1
1.0%
0.546
1
1.0%
0.548
1
1.0%
0.55
1
1.0%
0.551
1
1.0%
0.552
1
1.0%
0.554
4
3.9%
0.56
1
1.0%
0.569
1
1.0%
0.651
1
1.0%
0.661
1
1.0%
0.67
1
1.0%
0.672
1
1.0%
LARGEST VALUES
17.688
1
1.0%
6.684
1
1.0%
6.097
1
1.0%
5.658
1
1.0%
5.276
1
1.0%
5.032
1
1.0%
4.006
1
1.0%
3.495
1
1.0%
3.419
1
1.0%
3.29
1
1.0%
3.103
1
1.0%
2.827
1
1.0%
2.823
1
1.0%
2.424
1
1.0%
2.364
1
1.0%
peakExecutionMemory_GB
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
peakExecutionMemory
1.00
shuffleRecordsWritten
0.76
cpuTimeRatio
0.74
ioTimeRatio
-0.74
shuffleRemoteBytesRead
0.73
shuffleTotalBytesRead
0.73
shuffleTotalBytesRead_GB
0.73
shuffleRecordsRead
0.73
shuffleLocalBytesRead
0.73
shuffleBytesWritten_GB
0.71
shuffleBytesWritten
0.71
totalBytesWritten_GB
0.71
totalBytesWritten
0.71
executorCpuTime
0.70

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.51
memoryBytesSpilled
0.51
memoryBytesSpilledRatio
0.51
diskBytesSpilledRatio
0.42
AllSuccessful
0.26
aLogSuccess
0.25
FailureCounts
0.12
AppSuccessful
0.08
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.0
26
25.2%
12.549
3
2.9%
75.293
2
1.9%
12.486
2
1.9%
25.024
2
1.9%
656.496
2
1.9%
62.374
1
1.0%
595.08
1
1.0%
39.209
1
1.0%
38.567
1
1.0%
94.108
1
1.0%
221.462
1
1.0%
33.811
1
1.0%
42.465
1
1.0%
9.611
1
1.0%
SMALLEST VALUES
0.0
26
25.2%
0.19
1
1.0%
0.31
1
1.0%
9.611
1
1.0%
10.985
1
1.0%
12.111
1
1.0%
12.486
2
1.9%
12.549
3
2.9%
18.811
1
1.0%
23.195
1
1.0%
25.024
2
1.9%
25.379
1
1.0%
26.575
1
1.0%
26.899
1
1.0%
27.336
1
1.0%
LARGEST VALUES
914.014
1
1.0%
792.042
1
1.0%
700.391
1
1.0%
656.496
2
1.9%
595.08
1
1.0%
550.0
1
1.0%
435.887
1
1.0%
350.0
1
1.0%
303.125
1
1.0%
281.838
1
1.0%
237.095
1
1.0%
228.125
1
1.0%
221.462
1
1.0%
198.897
1
1.0%
178.128
1
1.0%
totalBytesRead_GB
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
totalBytesRead
1.00
shuffleTotalBytesRead_GB
0.90
shuffleTotalBytesRead
0.90
shuffleRemoteBytesRead
0.90
shuffleLocalBytesRead
0.90
shuffleBytesWritten_GB
0.85
shuffleBytesWritten
0.85
totalBytesWritten_GB
0.85
totalBytesWritten
0.85
shuffleWriteTime
0.84
executorRunTime
0.82
shuffleRecordsRead
0.82
executorCpuTime
0.78
shuffleRecordsWritten
0.78

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.42
memoryBytesSpilled
0.42
memoryBytesSpilledRatio
0.42
diskBytesSpilledRatio
0.42
AllSuccessful
0.26
aLogSuccess
0.25
AppSuccessful
0.06
FailureCounts
0.06
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.0
25
24.3%
4.207
1
1.0%
2.826
1
1.0%
37.81
1
1.0%
0.795
1
1.0%
11.741
1
1.0%
17.842
1
1.0%
60.985
1
1.0%
13.046
1
1.0%
0.425
1
1.0%
3.36
1
1.0%
120.013
1
1.0%
90.384
1
1.0%
9.835
1
1.0%
17.153
1
1.0%
SMALLEST VALUES
0.0
25
24.3%
0.005
1
1.0%
0.212
1
1.0%
0.425
1
1.0%
0.654
1
1.0%
0.672
1
1.0%
0.795
1
1.0%
1.436
1
1.0%
1.498
1
1.0%
1.983
1
1.0%
2.006
1
1.0%
2.024
1
1.0%
2.398
1
1.0%
2.826
1
1.0%
2.892
1
1.0%
LARGEST VALUES
549.766
1
1.0%
549.535
1
1.0%
536.858
1
1.0%
420.265
1
1.0%
317.06
1
1.0%
316.786
1
1.0%
282.646
1
1.0%
233.233
1
1.0%
209.516
1
1.0%
205.009
1
1.0%
167.254
1
1.0%
123.968
1
1.0%
120.013
1
1.0%
109.393
1
1.0%
91.42
1
1.0%
totalBytesWritten_GB
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
totalBytesWritten
1.00
shuffleBytesWritten_GB
1.00
shuffleBytesWritten
1.00
shuffleWriteTime
0.97
shuffleLocalBytesRead
0.93
shuffleTotalBytesRead
0.93
shuffleTotalBytesRead_GB
0.93
shuffleRemoteBytesRead
0.93
shuffleRecordsWritten
0.87
totalBytesRead_GB
0.85
totalBytesRead
0.85
executorCpuTime
0.79
ioTimeRatio
-0.77
cpuTimeRatio
0.77

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.29
memoryBytesSpilled
0.29
memoryBytesSpilledRatio
0.29
diskBytesSpilledRatio
0.29
AllSuccessful
0.22
aLogSuccess
0.21
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.0
27
26.2%
0.006
4
3.9%
0.001
2
1.9%
0.004
2
1.9%
0.01
1
1.0%
8.745
1
1.0%
28.055
1
1.0%
3.318
1
1.0%
1.786
1
1.0%
3.865
1
1.0%
1.158
1
1.0%
39.72
1
1.0%
1.14
1
1.0%
4.692
1
1.0%
7.282
1
1.0%
SMALLEST VALUES
0.0
27
26.2%
0.001
2
1.9%
0.004
2
1.9%
0.005
1
1.0%
0.006
4
3.9%
0.007
1
1.0%
0.008
1
1.0%
0.01
1
1.0%
0.014
1
1.0%
0.017
1
1.0%
0.033
1
1.0%
0.039
1
1.0%
0.058
1
1.0%
0.068
1
1.0%
0.078
1
1.0%
LARGEST VALUES
296.651
1
1.0%
230.492
1
1.0%
230.321
1
1.0%
230.089
1
1.0%
168.289
1
1.0%
165.642
1
1.0%
134.63
1
1.0%
79.659
1
1.0%
61.667
1
1.0%
57.603
1
1.0%
54.709
1
1.0%
43.37
1
1.0%
42.269
1
1.0%
41.128
1
1.0%
39.72
1
1.0%
shuffleTotalBytesRead_GB
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleTotalBytesRead
1.00
shuffleRemoteBytesRead
1.00
shuffleLocalBytesRead
1.00
shuffleBytesWritten_GB
0.93
shuffleBytesWritten
0.93
totalBytesWritten_GB
0.93
totalBytesWritten
0.93
shuffleRecordsRead
0.91
shuffleWriteTime
0.91
totalBytesRead
0.90
totalBytesRead_GB
0.90
shuffleRecordsWritten
0.85
executorCpuTime
0.80
peakExecutionMemory
0.73

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.49
memoryBytesSpilled
0.49
memoryBytesSpilledRatio
0.49
diskBytesSpilledRatio
0.49
AllSuccessful
0.19
aLogSuccess
0.18
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.0
27
26.2%
0.006
4
3.9%
0.001
2
1.9%
0.004
2
1.9%
8.865
1
1.0%
0.039
1
1.0%
43.37
1
1.0%
28.055
1
1.0%
3.318
1
1.0%
0.885
1
1.0%
0.307
1
1.0%
0.199
1
1.0%
0.017
1
1.0%
3.865
1
1.0%
0.005
1
1.0%
SMALLEST VALUES
0.0
27
26.2%
0.001
2
1.9%
0.004
2
1.9%
0.005
1
1.0%
0.006
4
3.9%
0.008
1
1.0%
0.01
1
1.0%
0.014
1
1.0%
0.017
1
1.0%
0.033
1
1.0%
0.039
1
1.0%
0.04
1
1.0%
0.045
1
1.0%
0.068
1
1.0%
0.078
1
1.0%
LARGEST VALUES
449.089
1
1.0%
448.858
1
1.0%
443.885
1
1.0%
309.522
1
1.0%
230.492
1
1.0%
165.642
1
1.0%
145.169
1
1.0%
92.876
1
1.0%
79.659
1
1.0%
61.033
1
1.0%
57.603
1
1.0%
43.37
1
1.0%
42.468
1
1.0%
42.269
1
1.0%
41.128
1
1.0%
shuffleBytesWritten_GB
MISSING:
---
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
shuffleBytesWritten
1.00
totalBytesWritten_GB
1.00
totalBytesWritten
1.00
shuffleWriteTime
0.97
shuffleLocalBytesRead
0.93
shuffleTotalBytesRead
0.93
shuffleTotalBytesRead_GB
0.93
shuffleRemoteBytesRead
0.93
shuffleRecordsWritten
0.87
totalBytesRead_GB
0.85
totalBytesRead
0.85
executorCpuTime
0.79
cpuTimeRatio
0.77
ioTimeRatio
-0.77

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
diskBytesSpilled
0.29
memoryBytesSpilled
0.29
memoryBytesSpilledRatio
0.29
diskBytesSpilledRatio
0.29
AllSuccessful
0.22
aLogSuccess
0.21
FailureCounts
0.05
AppSuccessful
0.05
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
0.0
27
26.2%
0.006
4
3.9%
0.001
2
1.9%
0.004
2
1.9%
0.01
1
1.0%
39.72
1
1.0%
3.318
1
1.0%
1.786
1
1.0%
3.865
1
1.0%
1.158
1
1.0%
8.745
1
1.0%
4.692
1
1.0%
1.14
1
1.0%
7.282
1
1.0%
0.005
1
1.0%
SMALLEST VALUES
0.0
27
26.2%
0.001
2
1.9%
0.004
2
1.9%
0.005
1
1.0%
0.006
4
3.9%
0.007
1
1.0%
0.008
1
1.0%
0.01
1
1.0%
0.014
1
1.0%
0.017
1
1.0%
0.033
1
1.0%
0.039
1
1.0%
0.058
1
1.0%
0.068
1
1.0%
0.078
1
1.0%
LARGEST VALUES
296.647
1
1.0%
230.492
1
1.0%
230.32
1
1.0%
230.089
1
1.0%
168.289
1
1.0%
165.642
1
1.0%
134.63
1
1.0%
79.659
1
1.0%
61.667
1
1.0%
57.603
1
1.0%
54.709
1
1.0%
43.37
1
1.0%
42.269
1
1.0%
41.128
1
1.0%
39.72
1
1.0%
sparkSetupTime
MISSING:
2
(2%)
NUMERICAL ASSOCIATIONS
(PEARSON, -1 to 1)
bytesWritten
0.66
recordsWritten
0.29
submissionTime
0.24
AppDurationSecs
0.19
resultSerializationTime
-0.18
elapsedTime_minutes
0.18
elapsedTime
0.18
executorRunTime
0.14
shuffleRemoteBlocksFetched
0.13
shuffleTotalBlocksFetched
0.13
shuffleLocalBlocksFetched
0.13
totalBytesWritten
0.12
totalBytesWritten_GB
0.12
shuffleBytesWritten
0.12

CATEGORICAL ASSOCIATIONS
(CORRELATION RATIO, 0 to 1)
AppSuccessful
0.38
FailureCounts
0.27
aLogSuccess
0.26
AllSuccessful
0.24
diskBytesSpilled
0.08
memoryBytesSpilled
0.08
memoryBytesSpilledRatio
0.08
diskBytesSpilledRatio
0.02
App Name
0.00
RunType
0.00
shuffleRemoteBytesReadToDisk
0.00
MOST FREQUENT VALUES
4.4239999999999995
2
2.0%
4.324000000000005
1
1.0%
4.719999999999999
1
1.0%
4.045999999999999
1
1.0%
4.6270000000000095
1
1.0%
4.445
1
1.0%
4.676000000000002
1
1.0%
4.231000000000002
1
1.0%
4.378999999999998
1
1.0%
5.06899999999996
1
1.0%
4.740000000000009
1
1.0%
4.685000000000002
1
1.0%
4.360999999999997
1
1.0%
4.890000000000001
1
1.0%
7.277999999999992
1
1.0%
SMALLEST VALUES
3.7069999999999936
1
1.0%
4.014000000000003
1
1.0%
4.045999999999999
1
1.0%
4.134
1
1.0%
4.174999999999997
1
1.0%
4.181999999999995
1
1.0%
4.192999999999998
1
1.0%
4.194000000000003
1
1.0%
4.202999999999996
1
1.0%
4.204999999999998
1
1.0%
4.231000000000002
1
1.0%
4.247
1
1.0%
4.249000000000002
1
1.0%
4.2719999999999985
1
1.0%
4.286999999999999
1
1.0%
LARGEST VALUES
18.745000000000005
1
1.0%
9.828000000000003
1
1.0%
9.230999999999995
1
1.0%
8.912999999999997
1
1.0%
7.836000000000013
1
1.0%
7.277999999999992
1
1.0%
7.206999999999994
1
1.0%
7.036000000000001
1
1.0%
6.797999999999973
1
1.0%
6.745999999999995
1
1.0%
6.648000000000003
1
1.0%
6.626999999999995
1
1.0%
6.454999999999984
1
1.0%
6.371000000000002
1
1.0%
6.3359999999999985
1
1.0%